{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adaline: Adaptive Linear Neuron Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An implementation of the ADAptive LInear NEuron, Adaline, for binary classification tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.classifier import Adaline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An illustration of the ADAptive LInear NEuron (Adaline) -- a single-layer artificial linear neuron with a threshold unit:\n",
    "    \n",
    "![](./Adaline_files/adaline_schematic.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Adaline classifier is closely related to the Ordinary Least Squares (OLS) Linear Regression algorithm; in OLS regression we find the line (or hyperplane) that minimizes the vertical offsets. Or in other words, we define the best-fitting line as the line that minimizes the sum of squared errors (SSE) or mean squared error (MSE) between our target variable (y) and our predicted output over all samples $i$ in our dataset of size $n$.\n",
    "\n",
    "$$ SSE =  \\sum_i (\\text{target}^{(i)} - \\text{output}^{(i)})^2$$\n",
    "\n",
    "$$MSE = \\frac{1}{n} \\times SSE$$\n",
    "\n",
    "\n",
    "[`LinearRegression`](../regressor/LinearRegression.md) implements a linear regression model for performing ordinary least squares regression, and in Adaline, we add a threshold function $g(\\cdot)$ to convert the continuous outcome to a categorical class label:\n",
    "\n",
    "$$y = g({z}) =\n",
    "\\begin{cases}\n",
    "1 & \\text{if z $\\ge$ 0}\\\\\n",
    "-1 & \\text{otherwise}.\n",
    "\\end{cases}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An Adaline model can be trained by one of the following three approaches:\n",
    "\n",
    "- Normal Equations\n",
    "- Gradient Descent\n",
    "- Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normal Equations (closed-form solution)\n",
    "\n",
    "The closed-form solution should be preferred for \"smaller\" datasets where calculating (a \"costly\") matrix inverse is not a concern. For very large datasets, or datasets where the inverse of $[X^T X]$ may not exist (the matrix is non-invertible or singular, e.g., in case of perfect multicollinearity), the gradient descent or stochastic gradient descent approaches are to be preferred.\n",
    "\n",
    "The linear function (linear regression model) is defined as:\n",
    "\n",
    "$$z = w_0x_0 + w_1x_1 + ... + w_mx_m = \\sum_{j=0}^{m} w_j x_j = \\mathbf{w}^T\\mathbf{x}$$\n",
    "\n",
    "where $y$ is the response variable, $\\mathbf{x}$ is an $m$-dimensional sample vector, and $\\mathbf{w}$ is the weight vector (vector of coefficients). Note that $w_0$ represents the y-axis intercept of the model and therefore $x_0=1$.  \n",
    "\n",
    "Using the closed-form solution (normal equation), we compute the weights of the model as follows:\n",
    "\n",
    "$$ \\mathbf{w} = (\\mathbf{X}^T\\mathbf{X})^{-1}\\mathbf{X}^Ty$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Descent (GD)  and Stochastic Gradient Descent (SGD) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the current implementation, the Adaline model is learned via Gradient Descent or Stochastic Gradient Descent.\n",
    "\n",
    "See [Gradient Descent and Stochastic Gradient Descent](https://sebastianraschka.com/faq/docs/gradient-optimization.html) and [Deriving the Gradient Descent Rule for Linear Regression and Adaline](https://sebastianraschka.com/faq/docs/linear-gradient-derivative.html) for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Random shuffling is implemented as:\n",
    "\n",
    "- for one or more epochs\n",
    "    - randomly shuffle samples in the training set\n",
    "        - for training sample *i*\n",
    "            - compute gradients and perform weight updates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- B. Widrow, M. E. Hoff, et al. [Adaptive switching circuits](https://www.rob.uni-luebeck.de/index.php?id=267). 1960."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 - Closed Form Solution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAGqBJREFUeJzt3XmYVPWd7/H3h24WpQGDqIigqPgYlccl7jHGLCYax2jUyYJ7zA2Oj2R0onGizs11ZsyNd3KTGa86Gcho4u4Yl6txGTE3GpNxwV1UxAXBRgRtBFlEoLu/94+qNgU01OmqU6eqTn9ez9PP013n1O98Tzd8+lfnfPtXigjMzCw/BtS7ADMzS5eD3cwsZxzsZmY542A3M8sZB7uZWc442M3McsbBbhWTdLqkPyXc9xJJNxQ/317SCkktta2w7yR9TtL8jI/5a0mXZnlMyzcHu21A0sOSlkgaXIvxI+KtiGiLiK5ajF+OpAMk3SdpqaT3Jc2Q9O161FJO8ZdnV/EXYc/HlfWuyxqbg93WIWk8cCgQwDF1LaYGJB0M/B74AzAB2BI4C/hKPesq47HiL8Kejyl9HUBSay0Ks8bkYLf1nQo8DvwaOK10g6QtJd0taZmkGcDO622/XFJ7cfvTkg7t7QCSxkuKnrApvkL4R0n/JWm5pOmSRpXsf5CkR4sz7Oclfa6K8/spcG1E/K+I6IiCpyPiGxupdbdifUslvSTpmJJtR0l6uVjz25LOL9l2tKTnis97VNKeJdv2kfRM8Xn/AQyp5EQkjZB0naT3JM2T9HeSBhS3nV78fv6zpPeBS9Z7bKmkOZI+XXy8XdK7kk4rc1hrAg52W9+pwI3FjyMkbVOy7SrgI2Bb4IziR6kngb2BkcBNwG8kJQ2tE4FvA1sDg4DzASRtB9wLXFoc93zgdklb9fXEJG0OHAzclnD/gcBvgenFur4H3Chp1+IuVwNnRsQwYCKFVwJI+hRwDXAmhVcEU4G7JQ2WNAj4v8D1xfP5DXBCX8+l6ApgBLATcBiFn13pJaUDgTnF2n9c8tgLxbpuAm4B9qfw6uVk4EpJbRXWYw3CwW4fk/QZYAfg1oh4GniDQuBSvNF5AvCjiFgZES8C15Y+PyJuiIjFEdEZET8DBgO7ksyvIuLViFgF3ErhFwQUwua+iLgvIroj4kHgKeCoCk7xExT+zb+TcP+DgDbgsohYExG/B+4BJhW3rwV2lzQ8IpZExDPFx78LTI2IJyKiKyKuBVYXxzsIGAj8S0SsjYjbKPxC3GQdxRl2z8dBxZ/HN4ELI2J5RMwFfgacUvK8BRFxRfHnsar42JsR8avi/Y3/AMYB/xARqyNiOrCGQshbE3OwW6nTgOkR0VH8+ib+fDlmK6AVaC/Zf17pkyWdJ2mWpA8kLaUwmxxFMgtLPv+QQqBC4RfN10uDDfgMhVcN65B0UskNxvt7OcYSoLu3527EGKA9IrpLHpsHbFf8/AQKv2DmSfpD8fp9T83nrVfzuOJ4Y4C3Y93V99b5Pvbi8YjYouTjcQrf10HrPbe0Nlj3Z9VjUcnnqwAiYv3HPGNvcr6hYgBI2gz4BtAiqSdkBwNbSNoLeBHopBBQrxS3b1/y/EOBvwW+CLwUEd2SlgCqsrR24PqI+G65HSOi5xLSxrZ/KOkxCoH8UIJjLwDGSRpQEu7bA68Wx3sSOLZ4yWYKhVca44o1/zgifrz+gJIOA7aTpJJw357Cq6O+6KDwimEH4OWScd4u2cdLt/ZTnrFbj68BXcDuFC6D7A3sBvwROLX40v0OCjfhNpe0O+veXB1GIfjfA1ol/QgYnkJdNwBflXSEpBZJQ1ToNR9b4XgXAKdL+oGkLQEk7SXpll72fQJYCVwgaWDxpu1XgVskDSq+QhgREWuBZRS+fwC/BP5K0oEqGCrpLyQNAx6j8H36a0mtko4HDujrSRR/HrcCP5Y0TNIOwPcpfL+sn3OwW4/TKFznfisiFvZ8AFcCJ6nQwTKFwsv0hRS6Zn5V8vwHgPspzGbnUbjJ2tulgD6JiHbgWOAiCr802oEfUOG/3Yh4FPhC8WNOsWNkGnBfL/uuodDy+RUKM+R/pfBLrucVyynAXEnLgL+icD+AiHiKwnX2Kylc/nkdOL1kzOOLXy+hcJ38jkrOhcLN3JUUbpD+icKls2sqHMtyRH6jDTOzfPGM3cwsZxzsZmY542A3M8sZB7uZWc7Up4/91em+Y2tm1hebj4Sx+yX6u5D6BHvH7Loc1sysaQ0fA2P3S7SrL8WYmeWMg93MLGcc7GZmOeNFwMys3+hGrGwZSVfrEKpfn65WgpbOjxja9T4DKlzHzcFuZv3GypaRDGzbgjZ1oQbN9QhYHUNYuQKGdS2uaAxfijGzfqOrdQiDGzjUASQYrK7iq4rKONjNrB9RQ4d6j0KNlRfqYDczyxkHu5lZxv7zj0+z61FnMeGIyVz2y0Tvrd4nDnYzswx1dXVx9qVTuX/q/+Dl317Fzfc9wsuvv5XqMdwVY2bWiwNOvpiOD1Zt8PioEZsx44YN3s42sRkzX2PC9tuy07jRAHzrK4dy1++fYPcJ25d5ZnIOdjOzXnR8sIo9zvznDR5/aerfVDXu24sWM270qI+/Hjt6FE+8kO76Wb4UY2aWod7ejlQp/7GUg93MLENjR4+ifWHHx1/PX9jBmK1HpnoMB7uZWYb2n7gLr81bwJvzF7JmzVpuuf+PHPP5A1M9hq+xm5llqLW1hSsvPpMjvnsJXd3dnHHc4eyxS3o3TsHBbmbWq1EjNuv1RumoEZtVPfZRh+3HUYcle9OMSjjYzcx6UU1LY735GruZWc442M3McsbBbmaWMw52M7OccbCbmeWMg93MLGNnXHw5W3/mFCYeM6Um4zvYzcwydvpxX+Q/p11Ss/Ed7GZmm9CxZBknTPkHFi9dltqYn91vIiNHtKU23voc7GZmm3DdHQ+w5O3Xufb2B+pdSmIOdjOzjehYsox7HnyIXxy/Dfc8+FCqs/ZacrCbmW3EdXc8wNE7i123GcLRO6tpZu0OdjOzXvTM1k/ddzgAp+47vGlm7VUHu6Rxkh6SNEvSS5LOSaMwM7N66pmtj2orrJU4qq01tVn7pPN/ysGTLmD23LcZ+/lvc/Xt06ses1Qaqzt2AudFxDOShgFPS3owIl5OYWwzs7p4eMbzLHhnNTfNfGedx8d0PM/3v/P1qsa++X//oKrnl1N1sEfEO8A7xc+XS5oFbAc42M2sad099dJ6l1CxVK+xSxoP7AM80cu2yZKekvTUtLv+K83DmplZidTeaENSG3A7cG5EbHB3ISKmAdMAePSKDd+m28ys5oIIkOpdx6ZFAFQek6nM2CUNpBDqN0bEHWmMaWaWtpbOj1gdLcXgbEwRsDpaaOn8qOIxqp6xSxJwNTArIn5e7XhmZrUytOt9Vq6Aj1qHAI06bQ9aOpcztOv9ikdI41LMIcApwExJzxUfuygi7kthbDOz1AwgGNa1GLrqXUltpdEV8yca91efmVm/4788NauhjqUrOOGH/8biD1bWuxTrRxzsZjV03b2PsmRhO9fe4xZfy46D3axGOpau4J4/PMkvjh/FPX940rN2y4yD3axGrrv3UY6eMIBdtx7M0RMGeNZumXGwm9VAz2z91E8NBeDUTw31rN0y42A3q4Ge2fo6KwN61m4ZSW1JATP7s4efeZUF767mppnvrvP4mEWv8v2Tvlynqqy/cLCb1cDdP5tS7xIaUsfSFZx52Q1Mu/AUthwxtN7l5JYvxZhZZtz+mQ0Hu5llwu2f2XGwm1km3P6ZHQe7mdWc2z+z5WA3s5pz+2e23BVjZjXn9s9sOdjNrObc/pktX4oxq7PZ8xax49cu4rX2d8vvXCEvH9y/ONjN6uyHV93GyNZVXHDFb2p2DPeP9y8OdrM6mj1vETNfeYNffW0oM195oyazdveP9z8OdrM6+uFVt3HixFb2HD2QEye21mTW7v7x/sfBblYnPbP1sw4YAsBZBwxJfdbu/vH+ycFuVic9s/Vt21oA2LatJfVZu/vH+ye3O5rVybOz25mxZi1XP7t0nccHDmpP7RjuH++fFBHZH/XRK+pwUDOzJjZ8DEw8QUl29aUYa0pp9GVn0dud5BhZnEu127OqM6sx8s7Bbk0pjb7sLHq7kxwji3OpdntWdWY1Rt452K3ppNGXnUVvd5JjZHEu1W7Pqs6sxugPHOzWdNLoy86itzvJMbI4l2q3Z1VnVmP0Bw52aypp9GVn0dud5BhZnEu127OqM6sx+gsHuzWVNPqys+jtTnKMLM6l2u1Z1ZnVGP2F+9itqaTRl51Fb3eSY2RxLtVuz6rOrMboL9zHbpnrWLqCMy+7gWkXnsKWI4bWu5yGN3veIo4853KmX3Euu4zbut7lbFSSn2u5fdL4t5Hbf1/uY7dG5na1vsliWd80pNEy6XbIdDjYLVNuV+ubLJb1TUMaLZNuh0yPg90y5Xa1vsliWd80pNEy6XbI9KQS7JKukfSupBfTGM/yye1qfZPFsr5pSKNl0u2Q6Uprxv5r4MiUxrKccrta32SxrG8a0miZdDtkulJpd4yIRySNT2Msyy+3q/VNFsv6piGNlkm3Q6YrtXbHYrDfExETN7J9MjAZYOoF39x38rGHpHJcM7N+oRHbHSNiWkTsFxH7OdQtC7PnLWLHr1200WvSj898k5Ff+htmvDx3o2Mk2afW0lhSN41le9Oo07LhrhjLrXL932f90/WMHx6c+ZPrNjpGkn1qLYv+cPeP54uD3XKpXP/34zPfZMnixVxz7GYsWby41xl5kn1qLYv+cPeP509a7Y43A48Bu0qaL+k7aYxrVqly/d9n/dP1nLTnQPYe3cpJew7sdUaeZJ9ay6I/3P3j+ZNKsEfEpIjYNiIGRsTYiLg6jXHNKlGu/7tnJn72/oMAOHv/QRvMyJPsU2tZ9Ie7fzyffCnGcqdc/3fPTHzMsML2McNaNpiRJ9mn1rLoD3f/eD552V7LnXL93+0Ll3D1om6ufmbtOts7Y8nHnyfZp9ay6A93/3g+edleM7Nm0Ih97GZZy6I3u1yvfBrcH2595WC33MqiNzuLtdLdH2595WC3XMqiNzuLtdLdH26VcLBbLmXRm53FWunuD7dKONgtd7Lozc5irXT3h1ulHOyWO1n0ZmexVrr7w61S7mO33MmiNzuLtdLdH26Vch+7mVkzcB+71VOj9F1nsQa5WSNysFvqGqXvOos1yM0akYPdUtUofddZrEFu1qgc7JaqRum7zmINcrNG5WC31DRK33UWa5CbNTIHu6WmUfqus1iD3KyRuY/dUtMofddZrEFu1sjcx27r6Fi6gjMvu4FpF57CliOG9rrP7HmLOPKcy5l+xbnsMm7rjCtMLsm5VDtGFscwA9zHbpVL0gKYxVK1achi2d4sjmHWVw52+1iSFsAslqpNQxbL9mZxDLNKONjtY0laALNYqjYNWSzbm8UxzCrhYDcgWQtgFkvVpiGLZXuzOIZZpRzsBiRrAcxiqdo0ZLFsbxbHMKuU2x0NSNaqmMVStWnIYtneLI5hVim3O5qZNYM+tDt6xm5mVkcfrFjFJbfMIFoGbXK/8TvuxLkTT0g0poPdzKyM9kVLWL22M9G+Nz3yGnNXDWGAEk2uWRsD2Of4ixg6fItN7jd6xJBE44GD3cz6mdnt7/HQzOT3heYu/ICFg8fTtuU2ifbfar+TOHin3SstLxUOdjNrau2LlvDTu5+ntXXTlzJ6LO4eym5HnAEJZ9Rb7T+InT8xqpoSM+dgN7Oaem/J8sSXMSLgX377Astahicev1OD2f/ESxk4eHClJeaOg93M+uTZ1xbwwtz3Eu379uIVvLRyBCO2GpN4/LGHn8Unx+5YaXmGg92sX1v8wUp+cttTxICWRPt3dwcfDBnDTp+elGj/1t0H87ltkoe6pcPBbtbgliz7kK7u7kT7rlnbxT/e9jTdgzfdYdHjoy7Y9xv/nc2GDqumRGswDnazDL02/z2efSP52jovzO3gLUYzJHHwil1PuJDhI5vrZp+lK5Vgl3QkcDnQAvx7RFyWxrhmjW7OgsVcdd+LqKX8sksR8G73cHY69PjE47dNGM6hvpRhfVR1sEtqAa4CvgTMB56UdHdEvFzt2GbVighWrlqTaN+u7m7+/pYZrGppSzz+RwzmgJMupXVgsla7TyYe2axyaczYDwBej4g5AJJuAY4FHOyWuhfnvMObC5eW37Ho/81cwNLNx9HSOjDR/hO+fDajxuxQaXlmDSGNYN8OKP0zrvnAgevvJGkyMBlg6gXfZPKxh6RwaGtmi95fxs/vepYg2R+KrFnbydKh4xmz55cSH2PMV7dmj623rbREs6aURrD39r9yg9UbI2IaMA3w6o5NpLOzK/G+K1at5uIbn4DByS5lrFg7gP0nXcLgzTavtDwz60UawT4fGFfy9VhgQQrjWg089+p8Fi9L9g49D7+8iDdWj2DQoIR/0acB7PmXFzFsi5FVVGhm1Uoj2J8EdpG0I/A28C3gxBTGtTJmt7/HNb9Lfitj2cqPWDtmXz6xwz6J9t/is1vz+bHjK6zOzOql6mCPiE5JU4AHKLQ7XhMRL1VdWT+zes1aLrr+UVYPSH5ZYiWbceCk/0lLq/8cwcz+LJVEiIj7gPvSGKuRzZ63iI4+vNHwrY/NZUnrVr3fhejFHkeew8httqusODOzon491Xt81nzuf3puon1XrFpNx+Y7suVOeyUef9u/OJ6Jox3UZpat3AT70uUf8nc3PYEGJb+UsXrzbdj7uEsSzahHS3wy4R+hmJnVU12C/elX3iq7z50z5vLW6uEMSPCn2gDdamHfb/yIzYclX8fZzCyP6hLs17+7c9l9tjzkSxy6w4QMqjEzy5e6BPsen/5yPQ5rZtYv5OYau9XeT6ZMYsWK5Rs83tY2jAuvvLkOFZlZbxzsltiKFcvZ6b9dscHjc/79e3Woxsw2JtmdSTMzaxoOdjOznHGwm5nljIPdzCxnfPPUEmtrG9brjdK2Nr/DvVkjcbBbYm5pNGsODnZLjfvczRqDg91S4z53s8bgm6dmZjnjYDczyxkHu5lZzjjYzcxyxjdPLTXuczdrDA52S6xcO+M7b82hszs22L78/fdSq+GsI/YmWlo2eFxdXfzigedSO45bN62ZOdgtsXLtjJ3dwQ5Trttg+7wrT02thmhpYdyUGzZ4vP3Kk1M7Brh105qbr7GbmeWMg93MLGcc7GZmOeNgNzPLGd88tcTKtTO2DlCvN0pbByi1GtTV1euNUnV1pXYMcOumNTdFbNieVmu/fGRO9gc1M2tio0cM4at7jUk0S/KMvQGU65lOo6c6izH++pgDe+1jbx0g/s/dTyQ6RqNwH7s1Mwd7AyjXM51GT3UWY2TRx54V97FbM/PNUzOznHGwm5nljIPdzCxnHOxmZjlT1c1TSV8HLgF2Aw6IiKfSKKq/KdcznUZPdRZjZNHHnhX3sVszq6qPXdJuQDcwFTg/abC7j93MrG8y62OPiFkAUvPNyLKSxvrh5cZI0j9ebowkfdtnfnlP1DJwg32iay1Tp7+QSp1p9Mpn0U/vPndrZJn1sUuaDEwGOPm8S/nsMZOyOnRdpbF+eLkxkvSPlxsjSd+2WgYy7nu9jHHFyanVmUavfBb99O5zt0ZWNtgl/Q4Y3cumiyPirqQHiohpwDTwpRgzs1oqG+wRcXgWhZiZWTrc7mhmljNVBbuk4yTNBw4G7pX0QDplmZlZpartirkTuDOlWnIpjfXDy42RpH+83BhJ+raja+3HN0pLRdfa1OpMo1c+i35697lbI/N67FVqlra3cnWm0ZZZbQ1mtnFejz1DzdL2Vq7ONNoyq63BzNLhm6dmZjnjYDczyxkHu5lZzjjYzcxyxjdPq9QsbW/l6kyjLbPaGswsHW53NDNrAn1pd/SlGDOznHGwm5nljIPdzCxnHOxmZjnjYDczyxkHu5lZzjjYzcxyxsFuZpYzDnYzs5xxsJuZ5YyD3cwsZxzsZmY542A3M8sZB7uZWc442M3McsbBbmaWMw52M7OccbCbmeWMg93MLGcc7GZmOeNgNzPLGQe7mVnOONjNzHLGwW5mljMOdjOznHGwm5nlTFXBLumnkl6R9IKkOyVtkVZhZmZWmWpn7A8CEyNiT+BV4MLqSzIzs2pUFewRMT0iOotfPg6Mrb4kMzOrRprX2M8A7t/YRkmTJT0l6alH7r45xcOamVmp1nI7SPodMLqXTRdHxF3FfS4GOoEbNzZOREwDpgH88pE5UVG1ZmZWVtlgj4jDN7Vd0mnA0cAXI8KBbWZWZ2WDfVMkHQn8LXBYRHyYTklmZlaNaq+xXwkMAx6U9Jykf0uhJjMzq0JVM/aImJBWIWZmlg7/5amZWc442M3McsbBbmaWM1VdY6/UqGGD6nFYM7OmtcXmAxPvK7eeJyNpcvGPrHIlj+eVx3MCn1ezqed5+VJMcpPrXUCN5PG88nhO4PNqNnU7Lwe7mVnOONjNzHLGwZ5c7q4BFuXxvPJ4TuDzajZ1Oy/fPDUzyxnP2M3McsbBbmaWMw72Psjjm3dL+rqklyR1S9qv3vVUS9KRkmZLel3SD+tdTxokXSPpXUkv1ruWtEgaJ+khSbOK//7OqXdNaZA0RNIMSc8Xz+vv61GHg71v8vjm3S8CxwOP1LuQaklqAa4CvgLsDkyStHt9q0rFr4Ej611EyjqB8yJiN+Ag4Oyc/KxWA1+IiL2AvYEjJR2UdREO9j7I45t3R8SsiJhd7zpScgDwekTMiYg1wC3AsXWuqWoR8Qjwfr3rSFNEvBMRzxQ/Xw7MArarb1XVi4IVxS8HFj8y71BxsFduk2/ebXWxHdBe8vV8chAWeSdpPLAP8ER9K0mHpBZJzwHvAg9GRObnVZdFwBpZWm/e3UiSnFNOqJfH3M/bwCS1AbcD50bEsnrXk4aI6AL2Lt6Du1PSxIjI9P6Ig309eXzz7nLnlCPzgXElX48FFtSpFitD0kAKoX5jRNxR73rSFhFLJT1M4f5IpsHuSzF9UPLm3cf4zbsb0pPALpJ2lDQI+BZwd51rsl5IEnA1MCsifl7vetIiaauebjlJmwGHA69kXYeDvW9y9+bdko6TNB84GLhX0gP1rqlSxRvbU4AHKNyMuzUiXqpvVdWTdDPwGLCrpPmSvlPvmlJwCHAK8IXi/6XnJB1V76JSsC3wkKQXKEw0HoyIe7IuwksKmJnljGfsZmY542A3M8sZB7uZWc442M3McsbBbmaWMw52M7OccbCbmeXM/wdaR4JIepdQ2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "from mlxtend.classifier import Adaline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "X = X[0:100] # class 0 and class 1\n",
    "y = y[0:100] # class 0 and class 1\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "\n",
    "ada = Adaline(epochs=30, \n",
    "              eta=0.01, \n",
    "              minibatches=None, \n",
    "              random_seed=1)\n",
    "ada.fit(X, y)\n",
    "plot_decision_regions(X, y, clf=ada)\n",
    "plt.title('Adaline - Closed Form')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Iteration: 30/30 | Cost 3.79 | Elapsed: 0:00:00 | ETA: 0:00:00"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHCpJREFUeJzt3XmYVPWd7/H3h242acAFEREUt3HjusQlGmM0cSNe45rNuMRseL2axIyJcck1ZMZcnWSyOOqTEaMm7uNovDFGo+aOW2IU911cQZBFQRAaEaH7O3/UaVNAQ52uOnWq+vTn9Tz9PN11Tv3O9zTNp391zrd/pYjAzMyKo1+jCzAzs2w52M3MCsbBbmZWMA52M7OCcbCbmRWMg93MrGAc7H2cpBMl/SXlvpMkXZN8vqmkdkkt9a2wcST9RtJ5yef7SJra6JrM0nCwF5SkeyUtkDSwHuNHxBsR0RYRHfUYvxJJu0m6LTnHhZKel/RjSevV43gR8UBEbJPFWJKmSTpgLdv3k9SZ/OJslzRT0o2Sds/i+PWQ/Lx9vdF1WImDvYAkjQP2AQI4rKHF1IGkjwH3An8Fto2IdYEJwApgpzU8pzW3ArMxKyLagKHAnsCLwAOS9m9sWdYbONiL6QTgIeA3wJfLN0jaQNKtkhZJmgJsucr2CyXNSLY/Jmmf7g4gaZyk6ArMZMb2z5L+KmmxpLskjSjbf09JDyaz66ck7VfD+f0EuDIizo+IufDhK4gfRsS9yfFOTGr5haR3gEmStpT0X5LmS5on6VpJ65bVuIukx5P6/wMYVLZtP0kzy74eLelmSW9Lel3St8q2TUpm2FclYz0nabdk29XApsAfktn4GWs70SiZGRHnAr8G/qXsONtKulvSO5KmSvp82bZDklcxiyW9Kem7ZdsOl/Rk8m/8qqQJyePDJV0uaXbynPO6LrV1XbKT9K/Jq6TXJX062fZjShOJi5NzujjtP6TVSUT4o2AfwCvA/wZ2BZYDG5VtuwG4ERgCjAfeBP5Stv04YAOgFTgdmAMMSrZNAq5JPh9H6RVBa/L1vcCrwD8Ag5OvL0i2bQLMBw6hNJk4MPl6wyrObQjQAexXYb8TKc3gv5mcy2Bgq+TYA4ENgfuBXyb7DwCmA98B+gOfTb535yXb9wNmJp/3Ax4Dzk2etwXwGnBw2ffp/eR8W4DzgYfKapsGHLCW2j881iqPfwroTL4HQ4AZwFeS8/sIMA/YIdl3NrBP8vl6wEeSz/cA3k2+D/2Sf5ttk23/D7g0GXskMAU4qez7uRz4RnJOJwOzAJX9+3+90T/7/ih9eMZeMJI+DmwG3BgRj1EK2y8l21qAo4FzI2JJRDwL/Lb8+RFxTUTMj4gVEfEzSiGY9trylRHxUkQspfTLY+fk8eOA2yPi9ojojIi7gUcpBV9PrUcpkOaUnfNPklcCSyT9oGzfWRFxUXIuSyPilYi4OyKWRcTbwM+BfZN996QU6L+MiOURcRPwyBpq2J3SL6V/iogPIuI14DLgi2X7/CU53w7gatZwiaiHZgEC1gUOBaZFxJXJ+T0O3EzpFxKUQnh7ScMiYkGyHeBrwBXJ96EzIt6MiBclbQR8Gjgt+dl4C/jFKuc0PSIuS87pt8DGwEYZnJdlzMFePF8G7oqIecnX1/H3yzEbUprdzSjbf3r5kyWdLukFSe9KWggMB0aQzpyyz98D2pLPNwM+l4TvwmTcj1MKhpVIOrbspuEd3RxjAaVZ64fPjYgzonSd/Zbk/LqUnyeSRkq6IbnMsAi4puzcRgNvRkT5qngrfW/KbAaMXuV8zmblkFv1ezEog+v8m1B6lbQwqeGjq9RwLDAq2fdoSr84p0u6T9JeyeNjKf2y7+6c+gOzy8a7lNLMfbVzioj3kk/bsKbT224o2VpIGgx8HmiR1PWfcCCwrqSdgGcpXZ4YS+lmHJSu93Y9fx/g+8D+wHMR0SlpAaVZYi1mAFdHxDcq7RgR1wLXrmX7EkkPA0cB91QabpWvz08e2zEi5ks6Aui6Hjwb2ESSysJ9U7oPwRnA6xGxdYXjp60rrSOBx5PvwQzgvog4sNsDRDwCHC6pP3AqpVdQYynVvmU3T5kBLANGRMSKKmrzMrFNxDP2YjmC0vXn7SldBtkZ2A54ADgheQn9O0o3EteRtD0r31wdSin43wZaJZ0LDMugrmuAz0g6WFKLpEHJzcgxVY53BvBVSWdKGgmQjLV5hecNBdqBhZI2Ab5Xtu1vlM79W5JaJR1F6Xp0d6YAiyR9X9Lg5JzGK3074lxK1+UrUskmkn4IfJ3SKwOA24B/kHS8pP7Jx+6StpM0IHnlMzwilgOLKP1cAFwOfEXS/pL6JWNvGxGzgbuAn0kalmzbUtK+q9ZU6zlZ/TnYi+XLlK5zvxERc7o+KM1Kj00uBZxK6eXzHEpdM1eWPf9O4A7gJUqXId5nlcsZ1YiIGcDhlELp7WTM71Hlz19E/IXSjcRPAC8llw3+ROkG3kVreeqPKN1kfBf4I6Vfcl1jfkDpVcCJlC73fKF8+yrH7wA+Q+kX5+uUblr+mtJlqzTOB36QXPL47hr2GS2pndIvokeA/0HphvFdSQ2LgYMoXQOfRenf818ovUIDOB6Yllxy+l+U7nMQEVMo3XD9RfJ9uI/SZRgodVMNAJ5Pvgc30c3lsjW4EPhs0jHzbymfY3WilS8pmplZb+cZu5lZwTjYzcwKxsFuZlYwDnYzs4JpTB/7S3f5jq2ZWU+ssz6M2S3V35Q0JtjneVlrM7MeGTYaxuyWaldfijEzKxgHu5lZwTjYzcwKxouAmVmf0YlY0rI+Ha2DqH1tu3oJWla8z5COd+hX5dpqDnYz6zOWtKxP/7Z1aVMHatJcj4BlMYgl7TC0Y35VY/hSjJn1GR2tgxjYxKEOIMFAdSSvKqrjYDezPkRNHepdSjVWX6iD3cysYBzsZmY5+9MDj7HNISez1cETueCymzIf38FuZpajjo4OTjnvUu649Ic8/4dLuP72+3n+lTcyPYa7YszMurHHcecw792lqz0+Yvhgplzz46rHnfLMy2y16cZsMbb0vuNf/PQ+/P6/Hmb7rTat8Mz0HOxmZt2Y9+5SdjjpF6s9/tyl36lp3DfnzmfsqBEffj1m1Agefjrb9bN8KcbMLEfdvR2pMv5jKQe7mVmOxowawYw58z78euaceYweuX6mx3Cwm5nlaPfxW/Py9Fm8PnMOH3ywnBvueIDDPvnRTI/ha+xmZjlqbW3h4nNO4uBvTKKjs5OvHnkAO2yd3Y1TcLCbmXVrxPDB3d4oHTF8cM1jH7Lvbhyyb7o3zaiGg93MrBu1tDQ2mq+xm5kVjIPdzKxgHOxmZgXjYDczKxgHu5lZwTjYzcxy9tVzLmTkx49n/GGn1mV8B7uZWc5OPHJ//jR5Ut3Gd7Cbma3FvAWLOPrUf2L+wkWZjfmJ3caz/vC2zMZblYPdzGwtrvrdnSx48xV+e/OdjS4lNQe7mdkazFuwiNvuvodfHbURt919T6az9npysJuZrcFVv7uTQ7cU22w0iEO3VK+ZtTvYzcy60TVbP2HXYQCcsOuwXjNrrznYJY2VdI+kFyQ9J+nbWRRmZtZIXbP1EW2ltRJHtLVmNms/5rs/Za9jzmDqtDcZ88mvcPnNd9U8ZrksVndcAZweEY9LGgo8JunuiHg+g7HNzBri3ilPMWv2Mq57ZvZKj4+e9xT/+LXP1TT29f/6vZqeX0nNwR4Rs4HZyeeLJb0AbAI42M2s17r10vMaXULVMr3GLmkcsAvwcDfbJkp6VNKjk3//1ywPa2ZmZTJ7ow1JbcDNwGkRsdrdhYiYDEwG4MGLVn+bbjOzugsiQGp0HWsXAVB9TGYyY5fUn1KoXxsRv8tiTDOzrLWseJ9l0ZIEZ3OKgGXRQsuK96seo+YZuyQBlwMvRMTPax3PzKxehnS8w5J2eL91ENCs0/agZcVihnS8U/UIWVyK2Rs4HnhG0pPJY2dHxO0ZjG1mlpl+BEM75kNHoyupryy6Yv5C8/7qMzPrc/yXp2Z1NG9hO0ef+e/Mf3dJo0uxPsTBblZHV/3xQRbMmcFvb3OLr+XHwW5WJ/MWtnPbfY/wq6NGcNt9j3jWbrlxsJvVyVV/fJBDt+rHNiMHcuhW/Txrt9w42M3qoGu2fsJHhgBwwkeGeNZuuXGwm9VB12x9pZUBPWu3nGS2pICZ/d29j7/ErLeWcd0zb630+Oi5L/GPxx7UoKqsr3Cwm9XBrT87tdElNKV5C9s56YJrmHzW8WwwfEijyyksX4oxs9y4/TMfDnYzy4XbP/PjYDezXLj9Mz8OdjOrO7d/5svBbmZ15/bPfLkrxszqzu2f+XKwm1nduf0zX74UY9ZgU6fPZfMjzublGW9V3rlKXj64b3GwmzXYmZfcxPqtSznjov+s2zHcP963ONjNGmjq9Lk88+KrXHnEEJ558dW6zNrdP973ONjNGujMS27iS+Nb2XFUf740vrUus3b3j/c9DnazBumarZ+8xyAATt5jUOazdveP900OdrMG6Zqtb9zWAsDGbS2Zz9rdP943ud3RrEGemDqDKR8s5/InFq70eP8BMzI7hvvH+yZFRP5HffCiBhzUzKwXGzYaxh+tNLv6Uoz1Sln0ZefR253mGHmcS63b86ozrzGKzsFuvVIWfdl59HanOUYe51Lr9rzqzGuMonOwW6+TRV92Hr3daY6Rx7nUuj2vOvMaoy9wsFuvk0Vfdh693WmOkce51Lo9rzrzGqMvcLBbr5JFX3Yevd1pjpHHudS6Pa868xqjr3CwW6+SRV92Hr3daY6Rx7nUuj2vOvMao69wH7v1Kln0ZefR253mGHmcS63b86ozrzH6CvexW+7mLWznpAuuYfJZx7PB8CGNLqfpTZ0+lwnfvpC7LjqNrceObHQ5a5Tm37XSPln8bBT258t97NbM3K7WM3ks65uFLFom3Q6ZDQe75crtaj2Tx7K+WciiZdLtkNlxsFuu3K7WM3ks65uFLFom3Q6ZnUyCXdIVkt6S9GwW41kxuV2tZ/JY1jcLWbRMuh0yW1nN2H8DTMhoLCsot6v1TB7L+mYhi5ZJt0NmK5N2x4i4X9K4LMay4nK7Ws/ksaxvFrJomXQ7ZLYya3dMgv22iBi/hu0TgYkAl57xhV0nHr53Jsc1M+sTmrHdMSImR8RuEbGbQ93yMHX6XDY/4uw1XpN+6JnXWf/A7zDl+WlrHCPNPvWWxZK6WSzbm0Wdlg93xVhhVer/PvknVzNuWHDS+VetcYw0+9RbHv3h7h8vFge7FVKl/u+HnnmdBfPnc8Xhg1kwf363M/I0+9RbHv3h7h8vnqzaHa8H/gZsI2mmpK9lMa5ZtSr1f5/8k6s5dsf+7DyqlWN37N/tjDzNPvWWR3+4+8eLJ5Ngj4hjImLjiOgfEWMi4vIsxjWrRqX+766Z+Cm7DwDglN0HrDYjT7NPveXRH+7+8WLypRgrnEr9310z8dFDS9tHD21ZbUaeZp96y6M/3P3jxeRle61wKvV/z5izgMvndnL548tX2r4iFnz4eZp96i2P/nD3jxeTl+01M+sNmrGP3SxvefRmV+qVz4L7w62nHOxWWHn0ZuexVrr7w62nHOxWSHn0ZuexVrr7w60aDnYrpDx6s/NYK9394VYNB7sVTh692Xmsle7+cKuWg90KJ4/e7DzWSnd/uFXLfexWOHn0ZuexVrr7w61a7mM3M+sN3MdujdQsfdd5rEFu1owc7Ja5Zum7zmMNcrNm5GC3TDVL33Uea5CbNSsHu2WqWfqu81iD3KxZOdgtM83Sd53HGuRmzczBbplplr7rPNYgN2tm7mO3zDRL33Uea5CbNTP3sdtK5i1s56QLrmHyWcezwfAh3e4zdfpcJnz7Qu666DS2Hjsy5wrTS3MutY6RxzHMAPexW/XStADmsVRtFvJYtjePY5j1lIPdPpSmBTCPpWqzkMeyvXkcw6waDnb7UJoWwDyWqs1CHsv25nEMs2o42A1I1wKYx1K1Wchj2d48jmFWLQe7AelaAPNYqjYLeSzbm8cxzKrldkcD0rUq5rFUbRbyWLY3j2OYVcvtjmZmvUEP2h09Yzcza6B325cy6YYpRMuAte43bvMtOG380anGdLCbmVUwY+4Cli1fkWrf6+5/mWlLB9FPqSbXLI9+7HLU2QwZtu5a9xs1fFCq8cDBbmZ9zNQZb3PPM+nvC02b8y5zBo6jbYONUu2/4W7HstcW21dbXiYc7GbWq82Yu4Cf3voUra1rv5TRZX7nELY7+KuQcka94e4D2HK9EbWUmDsHu5nV1dsLFqe+jBEBv/zD0yxqGZZ6/BUayO5fOo/+AwdWW2LhONjNrEeeeHkWT097O9W+b85v57klwxm+4ejU44854GS2HbN5teUZDnazPm3+u0s4/6ZHiX4tqfbv7AzeHTSaLT52TKr9W7cfyH4bpQ91y4aD3azJLVj0Hh2dnan2/WB5B/9802N0Dlx7h0WX9ztg18//HwYPGVpLidZkHOxmOXp55ts88Wr6tXWenjaPNxjFoNTBK7Y5+iyGrd+7bvZZtjIJdkkTgAuBFuDXEXFBFuOaNbvXZs3nktufRS2Vl12KgLc6h7HFPkelHr9tq2Hs40sZ1kM1B7ukFuAS4EBgJvCIpFsj4vlaxzarVUSwZOkHqfbt6OzkRzdMYWlLW+rx32cgexx7Hq3907XabZt6ZLPqZTFj3wN4JSJeA5B0A3A44GC3zD372mxen7Ow8o6J///MLBauM5aW1v6p9t/qoFMYMXqzasszawpZBPsmQPmfcc0EPrrqTpImAhMBLj3jC0w8fO8MDm292dx3FvHz3z9BkO4PRT5YvoKFQ8YxescDUx9j9GdGssPIjast0axXyiLYu/tfudrqjRExGZgMeHXHXmTFio7U+7YvXcY51z4MA9Ndymhf3o/dj5nEwMHrVFuemXUji2CfCYwt+3oMMCuDca0OnnxpJvMXpXuHnnufn8ury4YzYEDKv+hTP3b87NkMXXf9Gio0s1plEeyPAFtL2hx4E/gi8KUMxrUKps54myv+nP5WxqIl77N89K6st9kuqfZf9xMj+eSYcVVWZ2aNUnOwR8QKSacCd1Jqd7wiIp6rubI+ZtkHyzn76gdZ1i/9ZYklDOajx/xfWlr95whm9neZJEJE3A7cnsVYzWzq9LnM68EbDd/4t2ksaN2w+7sQ3dhhwrdZf6NNqivOzCzRp6d6D70wkzsem5Zq3/aly5i3zuZssMVOqcff+H8exfhRDmozy1dhgn3h4vf4wXUPowHpL2UsW2cjdj5yUqoZ9SiJbVP+EYqZWSM1JNgfe/GNivvcMmUabywbRr8Uf6oN0KkWdv38uawzNP06zmZmRdSQYL/6rS0r7rPB3geyz2Zb5VCNmVmxNCTYd/jYQY04rJlZn1CYa+xWf+efegzt7YtXe7ytbShnXXx9Ayoys+442C219vbFbPH1i1Z7/LVff7MB1ZjZmqS7M2lmZr2Gg93MrGAc7GZmBeNgNzMrGN88tdTa2oZ2e6O0rc3vcG/WTBzslppbGs16Bwe7ZcZ97mbNwcFumXGfu1lz8M1TM7OCcbCbmRWMg93MrGAc7GZmBeObp5YZ97mbNQcHu6VWqZ1x9huvsaIzVtu++J23M6vh5IN3JlpaVntcHR386s4nMzuOWzetN3OwW2qV2hlXdAabnXrVatunX3xCZjVESwtjT71mtcdnXHxcZscAt25a7+Zr7GZmBeNgNzMrGAe7mVnBONjNzArGN08ttUrtjK391O2N0tZ+yqwGdXR0e6NUHR2ZHQPcumm9myJWb0+rt8vufy3/g5qZ9WKjhg/iMzuNTjVL8oy9CVTqmc6ipzqPMb512Ee77WNv7Sf+7daHUx2jWbiP3XozB3sTqNQznUVPdR5j5NHHnhf3sVtv5punZmYF42A3MysYB7uZWcE42M3MCqamm6eSPgdMArYD9oiIR7Moqq+p1DOdRU91HmPk0ceeF/exW29WUx+7pO2ATuBS4Ltpg9197GZmPZNbH3tEvAAg9b4ZWV6yWD+80hhp+scrjZGmb/ukg3ZELf1X2yc6lnPpXU9nUmcWvfJ59NO7z92aWW597JImAhMBjjv9PD5x2DF5Hbqhslg/vNIYafrHK42Rpm9bLf0Z+81uxrjouMzqzKJXPo9+eve5WzOrGOyS/gyM6mbTORHx+7QHiojJwGTwpRgzs3qqGOwRcUAehZiZWTbc7mhmVjA1BbukIyXNBPYC/ijpzmzKMjOzatXaFXMLcEtGtRRSFuuHVxojTf94pTHS9G1Hx/IPb5SWi47lmdWZRa98Hv307nO3Zub12GvUW9reKtWZRVtmrTWY2Zp5PfYc9Za2t0p1ZtGWWWsNZpYN3zw1MysYB7uZWcE42M3MCsbBbmZWML55WqPe0vZWqc4s2jJrrcHMsuF2RzOzXqAn7Y6+FGNmVjAOdjOzgnGwm5kVjIPdzKxgHOxmZgXjYDczKxgHu5lZwTjYzcwKxsFuZlYwDnYzs4JxsJuZFYyD3cysYBzsZmYF42A3MysYB7uZWcE42M3MCsbBbmZWMA52M7OCcbCbmRWMg93MrGAc7GZmBeNgNzMrGAe7mVnBONjNzArGwW5mVjAOdjOzgqkp2CX9VNKLkp6WdIukdbMqzMzMqlPrjP1uYHxE7Ai8BJxVe0lmZlaLmoI9Iu6KiBXJlw8BY2ovyczMapHlNfavAnesaaOkiZIelfTo/bden+FhzcysXGulHST9GRjVzaZzIuL3yT7nACuAa9c0TkRMBiYDXHb/a1FVtWZmVlHFYI+IA9a2XdKXgUOB/SPCgW1m1mAVg31tJE0Avg/sGxHvZVOSmZnVotZr7BcDQ4G7JT0p6d8zqMnMzGpQ04w9IrbKqhAzM8uG//LUzKxgHOxmZgXjYDczK5iarrFXa8TQAY04rJlZr7XuOv1T7yu3nqcjaWLyR1aFUsTzKuI5gc+rt2nkeflSTHoTG11AnRTxvIp4TuDz6m0adl4OdjOzgnGwm5kVjIM9vcJdA0wU8byKeE7g8+ptGnZevnlqZlYwnrGbmRWMg93MrGAc7D1QxDfvlvQ5Sc9J6pS0W6PrqZWkCZKmSnpF0pmNricLkq6Q9JakZxtdS1YkjZV0j6QXkp+/bze6pixIGiRpiqSnkvP6USPqcLD3TBHfvPtZ4Cjg/kYXUitJLcAlwKeB7YFjJG3f2Koy8RtgQqOLyNgK4PSI2A7YEzilIP9Wy4BPRcROwM7ABEl75l2Eg70Hivjm3RHxQkRMbXQdGdkDeCUiXouID4AbgMMbXFPNIuJ+4J1G15GliJgdEY8nny8GXgA2aWxVtYuS9uTL/slH7h0qDvbqrfXNu60hNgFmlH09kwKERdFJGgfsAjzc2EqyIalF0pPAW8DdEZH7eTVkEbBmltWbdzeTNOdUEOrmMffzNjFJbcDNwGkRsajR9WQhIjqAnZN7cLdIGh8Rud4fcbCvoohv3l3pnApkJjC27OsxwKwG1WIVSOpPKdSvjYjfNbqerEXEQkn3Uro/kmuw+1JMD5S9efdhfvPupvQIsLWkzSUNAL4I3NrgmqwbkgRcDrwQET9vdD1ZkbRhV7ecpMHAAcCLedfhYO+Zwr15t6QjJc0E9gL+KOnORtdUreTG9qnAnZRuxt0YEc81tqraSboe+BuwjaSZkr7W6JoysDdwPPCp5P/Sk5IOaXRRGdgYuEfS05QmGndHxG15F+ElBczMCsYzdjOzgnGwm5kVjIPdzKxgHOxmZgXjYDczKxgHu5lZwTjYzcwK5r8BlDISlCaKGvMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Cost')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHYJJREFUeJzt3XuYHXWd5/H395w+fU3f0hdIOkl3gEgQhFwa5KaLqCwqK8RVWVdZZmUGZ55xBnUcB113B92dXRwv6M7uOIPCGl0UkatXFBGNIAN0rgQCJkA65EK6c+3uJN3py3f+qOqkE7s7pztdXeec+rye5zynTlWdOt+ioD/86lf1K3N3REQkuVJxFyAiIvFSEIiIJJyCQEQk4RQEIiIJpyAQEUk4BYGISMIpCEREEk5BICKScAoCEZGEK4q7gGzU19d7S0tL3GWIiOSVlStX7nL3hhOtlxdB0NLSQltbW9xliIjkFTNrz2Y9nRoSEUk4BYGISMIpCEREEk5BICKScAoCEZGEUxCIiCScgkBEJOEKOgh+9cJO/vHXm+IuQ0QkpxV0EDyxaTf/8Ogm9FxmEZGxFXQQtNSVc6h/kI7uvrhLERHJWQUdBM11FQBs3nUg5kpERHJXQQdBSxgE7bsPxlyJiEjuKuggmF1TSlHK2LxbLQIRkbEUdBAUpVPMnVmuFoGIyDgKOggAmuvK1SIQERlHwQdBS10F7bsP6hJSEZExJCAIyunpG2D3gcNxlyIikpMKPgia64evHNLpIRGR0RR8EAxfQvrKLnUYi4iMpuCDoKmmjHTK1CIQERlDZEFgZqVm9rSZrTWz58zsc+H8+Wb2lJltNLPvm1lxVDUAFBelaKopY7MuIRURGVWULYI+4HJ3Pw9YBFxpZhcCXwBuc/cFwF7ghghrAIJLSNUiEBEZXWRB4IGe8GMmfDlwOXBvOH85cE1UNQxrqavglV0HdAmpiMgoIu0jMLO0ma0BOoBHgJeAfe4+EK6yFWiKsgYIWgTdvQPsO9gf9U+JiOSdSIPA3QfdfREwB7gAOGu01Ub7rpndaGZtZtbW2dl5UnUMXzmkO4xFRP7QtFw15O77gF8DFwI1ZlYULpoDbB/jO7e7e6u7tzY0NJzU77fUaxRSEZGxRHnVUIOZ1YTTZcDbgA3AY8B7w9WuBx6KqoZhc2eWYaYWgYjIaIpOvMqkzQKWm1maIHDucfcfm9nzwN1m9j+A1cAdEdYAQElRmtnVZWoRiIiMIrIgcPd1wOJR5r9M0F8wrVrqy3lFTyoTEfkDBX9n8bDmugrdSyAiMorEBEFLXTl7D/azX5eQiogcIzFBMPwg+/Y9ahWIiIyUmCA4ei+BOoxFREZKTBDMm1kOQLs6jEVEjpGYICgrTnNqValaBCIix0lMEEBwCamuHBIROVaygqCuQi0CEZHjJCoImusq2NXTR3evLiEVERmWqCBoqQs7jNUqEBE5IlFBcOReAgWBiMgRCQuCoEWgUUhFRI5KVBBUlBTRUFmiK4dEREZIVBBA0E+gK4dERI5KXBBoFFIRkWMlLgha6srZ2dXHwcMDcZciIpITkhcE4fOLt+zR6SEREUhiEAyPQrpLQSAiAgkMgnm6hFRE5BiJC4Kq0gx1FcXqMBYRCSUuCCC4sUynhkREAokMghZdQioickQig6C5roLt+3vp7R+MuxQRkdglMgha6oMO41d1CamISDKDoFkPshcROSKRQTD/yHDU6icQEYksCMxsrpk9ZmYbzOw5M7spnH+LmW0zszXh651R1TCW6vIMNeUZ3UsgIgIURbjtAeCv3H2VmVUCK83skXDZbe7+pQh/+4Sa6yp0CamICBG2CNx9h7uvCqe7gQ1AU1S/N1HBcNRqEYiITEsfgZm1AIuBp8JZHzWzdWZ2p5nVTkcNx2uuq2D7vkP0DegSUhFJtsiDwMxmAPcBH3P3LuDrwOnAImAH8OUxvnejmbWZWVtnZ+eU19VSV86Qw9a9h6Z82yIi+STSIDCzDEEI3OXu9wO4+053H3T3IeAbwAWjfdfdb3f3VndvbWhomPLamnXlkIgIEO1VQwbcAWxw96+MmD9rxGrLgPVR1TCeluFRSNVhLCIJF+VVQ5cA1wHPmtmacN5ngA+Y2SLAgc3ARyKsYUwzK4qpLClSi0BEEi+yIHD3xwEbZdFPo/rNiTAzmuv1IHsRkUTeWTxMo5CKiCgI2Lr3EP2DQ3GXIiISm0QHQXNdOQNDzjZdQioiCZboIGipHx6FVKeHRCS5Eh0EzeElpO3qMBaRBEt0EDTMKKG8OK0WgYgkWqKDwMxorqtQi0BEEi3RQQAahVREJPFB0FxXwat7DjI45HGXIiISi8QHwfz6cvoHne37dAmpiCRT4oPg6Cik6icQkWRKfBC0hEHwivoJRCShEh8EjZUllGZStO9SEIhIMiU+CFIpo3lmhUYhFZHESnwQQHCHsUYhFZGkUhAQjDnUvucgQ7qEVEQSSEFA0CI4PDDEa129cZciIjLtFAQcvXJIdxiLSBIpCDg6HPXLnQoCEUkeBQEwu7qUmRXFrH11X9yliIhMOwUBwSikS+bVsnLL3rhLERGZdgqC0JLmGl7uPMCeA4fjLkVEZFopCEJL59UCsFqtAhFJGAVB6Nw5NRSljJXtCgIRSRYFQaisOM3Zs6sUBCKSOAqCEZY017J26z76B4fiLkVEZNpEFgRmNtfMHjOzDWb2nJndFM6faWaPmNnG8L02qhomamlzLb39Q2zY0RV3KSIi0ybKFsEA8FfufhZwIfDnZvZ64GbgUXdfADwafs4JS5uDTNLpIRFJksiCwN13uPuqcLob2AA0AVcDy8PVlgPXRFXDRM2qLmN2dSmrtujGMhFJjmnpIzCzFmAx8BRwirvvgCAsgMYxvnOjmbWZWVtnZ+d0lAkE/QSr1CIQkQSJPAjMbAZwH/Axd8/65Lu73+7ure7e2tDQEF2Bx1kyr5Zt+w6xY78eZi8iyRBpEJhZhiAE7nL3+8PZO81sVrh8FtARZQ0TNdxPsKpdp4dEJBmivGrIgDuADe7+lRGLfghcH05fDzwUVQ2T8frZVZRmUuowFpHEKIpw25cA1wHPmtmacN5ngFuBe8zsBmAL8L4Ia5iwTDrFuXNqNACdiCRGVkFgZt9x9+tONG8kd38csDEWvzX7Eqff0uZavrHiZXr7BynNpOMuR0QkUtmeGjp75AczSwNLp76c3LB0Xi0DQ866rfvjLkVEJHLjBoGZfdrMuoFzzawrfHUTdPDm1Ln9qbREN5aJSIKMGwTu/r/cvRL4ortXha9Kd69z909PU43TbmZFMafVV7BK/QQikgDZnhr6sZlVAJjZh8zsK2bWHGFdsRu+sczd4y5FRCRS2QbB14GDZnYe8CmgHfh2ZFXlgCXzatl94DDtuw/GXYqISKSyDYIBD/7X+Grga+7+NaAyurLipwHoRCQpsg2CbjP7NMF9AT8JrxrKRFdW/BY0zqCypEj3E4hIwcs2CK4F+oAPu/trBKOIfjGyqnJAKmUs1gB0IpIAWQVB+Mf/LqDazK4Cet29oPsIILif4MWd3XT19sddiohIZLIKAjN7P/A0wXAQ7weeMrP3RllYLljaXIs7rNHzCUSkgGU71tB/Ac539w4AM2sAfgncG1VhueC8udWkLOgwfvPrpm8obBGR6ZRtH0FqOARCuyfw3bxVWZrhzFOrdGOZiBS0bFsED5vZz4HvhZ+vBX4aTUm5ZWlzDQ+t3s7gkJNOjTWGnohI/jrRWENnmNkl7v7XwD8D5wLnAU8Ct09DfbFbMq+W7r4BNnZ0x12KiEgkTnR656tAN4C73+/un3D3jxO0Br4adXG5QDeWiUihO1EQtLj7uuNnunsb0BJJRTlm3sxy6mcUKwhEpGCdKAhKx1lWNpWF5CozY8k83VgmIoXrREHwjJn9yfEzw8dMroympNyztLmWzbsPsqunL+5SRESm3ImuGvoY8ICZfZCjf/hbgWJgWZSF5ZLhfoJV7Xu54uxTY65GRGRqjRsE7r4TuNjM3gKcE87+ibv/KvLKcsg5TdVk0sbKLQoCESk8Wd1H4O6PAY9FXEvOKs2kOaepWv0EIlKQCv7u4KmydF4t67bu5/DAUNyliIhMKQVBlpY019I3MMTzO7riLkVEZEopCLKkG8tEpFApCLJ0SlUpTTVl6icQkYITWRCY2Z1m1mFm60fMu8XMtpnZmvD1zqh+PwpLm2tpa99D8PhmEZHCEGWL4FvAlaPMv83dF4WvvBrBdGlzLTu7+ti+vzfuUkREpkxkQeDuK4A9UW0/DuonEJFCFEcfwUfNbF146qg2ht+ftIWnVlKWSaufQEQKynQHwdeB04FFwA7gy2OtaGY3mlmbmbV1dnZOV33jKkqnWDS3hrb2gmroiEjCTWsQuPtOdx909yHgG8AF46x7u7u3untrQ0PuPC/4kjPqWL+ti237DsVdiojIlJjWIDCzWSM+LgPWj7Vurrp6URMAD67eFnMlIiJTI8rLR79H8EjLM81sazh09d+b2bNmtg54C/DxqH4/KnNnlnN+Sy0PrN6my0hFpCBk+/D6CXP3D4wy+46ofm86LVs8h8888Czrt3XxhjnVcZcjInJSdGfxJLzrDbMoTqe4f/XWuEsRETlpCoJJqC7PcPnCRn60djsDgxqNVETym4JgkpYtaWJXz2F+u2lX3KWIiJwUBcEkveXMRmrKMzywSlcPiUh+UxBMUnFRiqvOncUvnn+Nnr6BuMsREZk0BcFJWLZ4Dr39Q/zs2R1xlyIiMmkKgpOwZF4NLXXlPKCby0QkjykIToKZcc3iJp58eTc79mvICRHJTwqCk3TNoibc4cHV2+MuRURkUhQEJ6mlvoIl82p4YPVWDTkhInlJQTAFli2Zw+939vD8jq64SxERmTAFwRS46g2zyKRN9xSISF5SEEyB2opi3nJmIw9pyAkRyUMKginyniVNdHb38cRLu+MuRURkQhQEU+QtCxupLsvwwCqNSCoi+UVBMEVKitK869xZ/Py5nRzQkBMikkcUBFNo2eImDvUP8vD61+IuRUQkawqCKdTaXMvcmWU8uEZXD4lI/lAQTCEzY9miJp7YtIudXb1xlyMikhUFwRRbtmQOQw4PqVUgInlCQTDF5tdXsGhuDffr5jIRyRMKggi8Z0kTL7zWzQYNOSEieUBBEIGrzp1NUcr0nAIRyQsKggjMrCjmsjMbeGjNNgaHNCKpiOQ2BUFEli2ew86uPn67sTPuUkRExqUgiMhbz2rk1KpSvvSLF9UqEJGcFlkQmNmdZtZhZutHzJtpZo+Y2cbwvTaq349baSbNp9+5kPXbuvhB26txlyMiMqYoWwTfAq48bt7NwKPuvgB4NPxcsN593mxam2v54s9fZP+h/rjLEREZVWRB4O4rgD3Hzb4aWB5OLweuier3c4GZccu7z2bPwcP870c3xl2OiMiopruP4BR33wEQvjdO8+9Pu3Oaqrm2dS7Lf7eZTR3dcZcjIvIHcraz2MxuNLM2M2vr7MzvK28++W/PpKw4zed/vEEPuBeRnDPdQbDTzGYBhO8dY63o7re7e6u7tzY0NExbgVGon1HCTW9dwIrfd/LohjF3WUQkFtMdBD8Erg+nrwcemubfj831F7dwekMF//0nz9M3MBh3OSIiR0R5+ej3gCeBM81sq5ndANwKvN3MNgJvDz8nQiad4r/9u7Np332QOx/fHHc5IiJHFEW1YXf/wBiL3hrVb+a6f/O6Bt52ViP/51cb+fdLmmisKo27JBGR3O0sLlSffdfr6R90vvDwi3GXIiICKAimXUt9BR++dD73rdrK6i174y5HRERBEIePXn4GjZUl3PKj5xnSOEQiEjMFQQxmlBTxN1cuZO2r+7hfzywQkZgpCGKybHETi+bW8IWHX6CnbyDuckQkwRQEMUmlgnGIOrv7+IdfaRwiEYmPgiBGi+bW8N6lc7jz8Vd4ZdeBuMsRkYRSEMTsU1eeSUlRmr/94XN6gI2IxEJBELPGylL+5h0LWfH7Tj774HoNSici0y6yO4sle9dd2Mxr+w/xfx97ieqyDDe/Y2HcJYlIgigIcsQnrziTrkMD/NNvgjD4s8tOj7skEUkIBUGOMDM+9+6z6ert5wsPv0BVWREffGNz3GWJSAIoCHJIKmV86X3n0d07wGcfXE9laYZ3nzc77rJEpMCpszjHZNIp/vGDSzi/ZSaf+P4aHntBD7IRkWgpCHJQaSbNHde3snBWJX/6/1fy9Ct74i5JRAqYgiBHVZZmWP6fL2BObRk3fOsZ1m/bH3dJIlKgFAQ5rG5GCd+54Y1UlWW4/s6neamzJ+6SRKQAKQhy3OyaMr5zwwWYwXXffIpt+w7FXZKIFBgFQR44rWEGyz98Ad19A1z3zafYsV9hICJTR0GQJ86eXc3/+6Pzea2rlytuW8EP2l7VcBQiMiUUBHmktWUmP/3LN7Hw1Er++t513LC8jdf298ZdlojkOQVBnmmpr+D7N17Ef73q9fzupV1ccdtvuHflVrUORGTSFAR5KJUybrh0Pj+76c287pRKPvmDtfzx8jZ2dql1ICITpyDIY/PrK/j+Ry7is+86i8c37eLtX/kN969S60BEJkZBkOfSKeOP33QaP7vpTSw4pZJP3LOWP/l2Gx1qHYhIlhQEBeK0hhncE7YOfrtxF2+/bQV3P72FvoHBuEsTkRxncZxGMLPNQDcwCAy4e+t467e2tnpbW9t0lFYQXurs4VP3rmNl+15mVhRz7flz+eAb5zGntjzu0kRkGpnZyhP9fYV4g6DV3Xdls76CYOLcnSc27ebbT27mlxt2AnD5wkauu6iFN51RTypl8RYoIpHLNgj0PIICZWZcuqCeSxfUs33fIb771BbufmYLv9zwNC115Xzowmbet3Qu1eWZuEsVkZjF1SJ4BdgLOPDP7n77KOvcCNwIMG/evKXt7e3TW2QB6hsY5OH1r/GdJ9tpa99LaSbF1ec1cd1FzZw9uwoztRJECkmunxqa7e7bzawReAT4C3dfMdb6OjU09Z7f3sV3/qWdB1dv41D/IE01ZVx8eh2XnFHPxafX0VhVGneJInKScjoIjinA7Bagx92/NNY6CoLodPX286O12/nt73fx5Mu72X+oH4AzGmdw8el1XHx6PReeNpOa8uKYKxWRicrZIDCzCiDl7t3h9CPA59394bG+oyCYHoNDzoYdXTyxaRe/e2k3T7+yh0P9g5jBObOrufj0OlpbZvK6U2Ywp7actDqcRXJaLgfBacAD4cci4Lvu/nfjfUdBEI/DA0Os3brvSDCs3rKX/sHg35eSohSnNczgjMYZLGg8+t5cV0FxkW5PEckFORsEk6EgyA0HDw+wYUc3L3X0sLGjm00dPWzs6GHr3qPPR0injJa6cs5onMGs6jJOqSrllKqSI++NVaVUlhSpY1pkGujyUZly5cVFLG2uZWlz7THzDx4e4OXOA2EwBAGxqaOHJzbtpqdv4A+2U5pJBcFQWUpjVQn1M0qoKstQVVoUvmeoKiuiqjRDdVmGqrIMlSVFuvdBJCIKAjlp5cVFnNNUzTlN1X+w7EDfAB3dfezs6mVnVy8dXeF0dx8dXb08t72LXd19dI8SGCOZwYziIspL0pRl0pRm0pQVB9NlmTSlI6bLitOUFqXIpFNkilIUH3k3MukUxeGy4nTwXpQ2ilJGOmUUpVKkw+ngc/ieNtJmpFJGyoyUcWQ6bYYZwXQqWKYWj+QTBYFEqqKkiPklRcyvrxh3vcEhp6dvgK5D/XT19tN1aCB876er9+j8Q4cHOdQ/eOS9t3/wyPze/iEOHh4I5w9N0x6ObTgcjOAdIwgJjoaFAQSLgs8jp8NtwNH5HJkXbOfo9PCyYwNo5MeR3xtt+cjtjLqtrHc82xWzX7UQgzXbPfqf73kD57fMjLQWBYHkhHTKqC4LTgVNBXdnYMjpHxyif8A5PDjE4cEh+geG6A+nDw8Er0F3BoeC9QcHw/chZ2BoiCF3BgaPLnd3hjwIriEffhG8Dx1d5mENHi5zgneGPzsMOTjB9HDNwfeOzh/+HEwxYt1gnWPmwXGfR/T/HbfO8O8d88/smGWMuexE/9yzlfWaud+NOWE+gZ0qy6QjrCSgIJCCZGZkwlNB6BYIkXHpOj8RkYRTEIiIJJyCQEQk4RQEIiIJpyAQEUk4BYGISMIpCEREEk5BICKScHkx+qiZdQKTfVZlPbBrCsvJBYW2T4W2P1B4+1Ro+wOFt0+j7U+zuzec6It5EQQnw8zashmGNZ8U2j4V2v5A4e1Toe0PFN4+ncz+6NSQiEjCKQhERBIuCUFwe9wFRKDQ9qnQ9gcKb58KbX+g8PZp0vtT8H0EIiIyviS0CEREZBwFHQRmdqWZvWhmm8zs5rjrOVlmttnMnjWzNWbWFnc9k2Fmd5pZh5mtHzFvppk9YmYbw/fa8baRS8bYn1vMbFt4nNaY2TvjrHGizGyumT1mZhvM7Dkzuymcn5fHaZz9ydvjZGalZva0ma0N9+lz4fz5ZvZUeIy+b2ZZPY2jYE8NmVka+D3wdmAr8AzwAXd/PtbCToKZbQZa3T1vr302szcDPcC33f2ccN7fA3vc/dYwsGvd/W/irDNbY+zPLUCPu38pztomy8xmAbPcfZWZVQIrgWuAPyIPj9M4+/N+8vQ4WfDszgp37zGzDPA4cBPwCeB+d7/bzP4JWOvuXz/R9gq5RXABsMndX3b3w8DdwNUx15R47r4C2HPc7KuB5eH0coL/SPPCGPuT19x9h7uvCqe7gQ1AE3l6nMbZn7zlgZ7wYyZ8OXA5cG84P+tjVMhB0AS8OuLzVvL84BMc6F+Y2UozuzHuYqbQKe6+A4L/aIHGmOuZCh81s3XhqaO8OIUyGjNrARYDT1EAx+m4/YE8Pk5mljazNUAH8AjwErDP3QfCVbL+m1fIQWCjzMv382CXuPsS4B3An4enJST3fB04HVgE7AC+HG85k2NmM4D7gI+5e1fc9ZysUfYnr4+Tuw+6+yJgDsEZkLNGWy2bbRVyEGwF5o74PAfYHlMtU8Ldt4fvHcADBAe/EOwMz+MOn8/tiLmek+LuO8P/SIeAb5CHxyk873wfcJe73x/OztvjNNr+FMJxAnD3fcCvgQuBGjMrChdl/TevkIPgGWBB2IteDPwH4Icx1zRpZlYRdnRhZhXAFcD68b+VN34IXB9OXw88FGMtJ234j2VoGXl2nMKOyDuADe7+lRGL8vI4jbU/+XyczKzBzGrC6TLgbQR9H48B7w1Xy/oYFexVQwDh5WBfBdLAne7+dzGXNGlmdhpBKwCgCPhuPu6PmX0PuIxgpMSdwN8CDwL3APOALcD73D0vOmDH2J/LCE43OLAZ+MjwufV8YGaXAr8FngWGwtmfITivnnfHaZz9+QB5epzM7FyCzuA0wf/Q3+Punw//TtwNzARWAx9y974Tbq+Qg0BERE6skE8NiYhIFhQEIiIJpyAQEUk4BYGISMIpCEREEk5BIIlgZj3he4uZ/ccp3vZnjvv8u6ncvkjUFASSNC3AhIIgHMl2PMcEgbtfPMGaRGKlIJCkuRV4Uzj+/MfDgbu+aGbPhIOPfQTAzC4Lx7D/LsGNSJjZg+GAf88ND/pnZrcCZeH27grnDbc+LNz2egueI3HtiG3/2szuNbMXzOyu8O5XzOxWM3s+rCXvhkeW/FR04lVECsrNwCfd/SqA8A/6fnc/38xKgCfM7BfhuhcA57j7K+HnD7v7nvCW/mfM7D53v9nMPhoO/nW89xDcuXoewZ3Hz5jZinDZYuBsgrFgngAuMbPnCYY6WOjuPjyEgEjU1CKQpLsC+E/hcL5PAXXAgnDZ0yNCAOAvzWwt8C8EAxouYHyXAt8LBzbbCfwGOH/EtreGA56tIThl1QX0At80s/cAB09670SyoCCQpDPgL9x9Ufia7+7DLYIDR1Yyu4xgYK+L3P08gnFcSrPY9lhGjv8yCBSF48hfQDBK5jXAwxPaE5FJUhBI0nQDlSM+/xz4s3CYYszsdeHorserBva6+0EzW0gw5O+w/uHvH2cFcG3YD9EAvBl4eqzCwvHyq939p8DHCE4riUROfQSSNOuAgfAUz7eArxGcllkVdth2Mvrj/R4G/tTM1gEvEpweGnY7sM7MVrn7B0fMfwC4CFhLMMLlp9z9tTBIRlMJPGRmpQStiY9PbhdFJkajj4qIJJxODYmIJJyCQEQk4RQEIiIJpyAQEUk4BYGISMIpCEREEk5BICKScAoCEZGE+1f7AfVD2wKXTgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "from mlxtend.classifier import Adaline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "X = X[0:100] # class 0 and class 1\n",
    "y = y[0:100] # class 0 and class 1\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "\n",
    "ada = Adaline(epochs=30, \n",
    "              eta=0.01, \n",
    "              minibatches=1, # for Gradient Descent Learning\n",
    "              random_seed=1,\n",
    "              print_progress=3)\n",
    "\n",
    "ada.fit(X, y)\n",
    "plot_decision_regions(X, y, clf=ada)\n",
    "plt.title('Adaline - Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(ada.cost_)), ada.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - Stochastic Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Iteration: 15/15 | Cost 3.81 | Elapsed: 0:00:00 | ETA: 0:00:00"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHndJREFUeJzt3XmcVNWZ//HPQ3ezNouAiGwKSnBhlCiijomauBHHuCVGiWKIGviZYKJjxrhkEvMbE00mY34O+spAojEu0WRcosEo4G9URKOoSERRXFBodkG2RrbufuaPe1uLprqruuv0rerb3/fr1a9XVd1T5zz3VvfTp+596pS5OyIikh4dih2AiIiEpcQuIpIySuwiIimjxC4ikjJK7CIiKaPELiKSMkrsRWRmE8xsTp5trzeze+LbQ8ys2szKWjfC4jGzO83shgTGOd/MZrb2OC2VeRzM7PNmtqjYMUnpU2JvBWb2tJmtN7NOrdG/uy9190p3r22N/ptiZoPM7EEzW2tmG81sgZlNiLfta2ZuZuVJx5WPbPG5+73ufnIL+xttZtPj13qDmS00s5+a2R7hov6Uuz/r7iNC9GVmH5jZiU1sP97M6uIJRLWZLTOzP5nZESHGbw3x390lxY6jFCixB2Zm+wKfBxw4vajBtI67gSpgH6APcCGwuqgRFYGZ/SPwNPAccIC79wLGAjXAoY08pyT/4TVhhbtXAt2Bo4C3gGfN7ITihiU5ubt+Av4APyL6Y78ZmN5gWx/gUWATMBf4N2BOxvZbiJLmJuAV4PMZ264H7olv70v0j6M8vv903NdzwGZgJtA347lHAc8DG4C/A8cXsH/VwKhGti2N46qOf44mmjz8EFgCrAHuAnpmPOdzGbFVARPix+8EbgMei/fpRWC/PI/VGODleNtq4OYm4pvQ4DU4GJgFfBQ/99pG9nUOMCXHsZoQvya/ivu7AdgP+B9gHbAWuBfolfGczwLz4n3+I3A/cEO87XhgWUbbAcCDwIfA+8B3G/y+/Ck+3puBN4DR8ba7gTpga3wcrsoS+y5jZTx+K/Byxv0DMo7XIuBrGdtOBRbG4y8Hvp+x7QxgfvwavQeMjR/vCdwOrIyfcwNQlnE85wC/BNbH+/yleNtPgVpgW7xPtxY7FxTzp+gBpO0HeBf4NnA4sBPYK2Pb/fEfWzdgZPyLm5lULiBK/uXAlcAqoHO87XqaTuzvAZ8BusT3b4q3DYyTyKlESfak+P6eLdy/J4mS1XnAkAbbdokrfuyi+JgMAyqBh4C7421D4j/6cUBFvO+j4m13xsliTHw87gXuz/NY/Q0YH9+uBI5qIr4J9a8B0cx0Zdxf5/j+kVmOQbc4iRyf41hNIJrBXxbH2QXYP34NOgF7ArOB/xe370j0D/CK+Hh8Nf4d2i2xx6/lK0QTiY7x8V0MnJLx+7Itft3LgBuBFzJi+wA4sYnYPxmrweNfJPqn0C3+qQK+Ge/fYUT/rA6O264k/ocL7AEcFt8eA2yMj0MHot/RA+Jtfwamxn33I5oATco4njuBb8X7dCmwArCMv4NLip0DSuGn6AGk6Ydo9rmTeLZM9Nb1ivh2WbztgIz2PyMjsWfpbz1waHz7eppO7D/MeN63gSfi2z8gTqQZ22cA32jhPu4B3EQ0A6wlmnUdkS2u+LH/D3w74/6I+DiUA9cADzcyzp3AbzPunwq8leexmg38hIx3LU3EN4FPE/s44NU8jsGguJ/M1/IXRO86ttS/FnHfS3P0dWb9mMCxmYkqfux5sif2Ixv2HR/P32X8vjyZse0gYGvG/Q9oWWI/IN73gcC5wLMNtk8FfhzfXgpMAnpkafOrLH3vBWwHumQ8Ng54KuN4vpuxrWscS/+MvwMldnedYw/sG8BMd18b3/9D/BhEs7NyohlOvSWZTzazK83szfii5Aait6V98xx7Vcbtj4lmqhCdCz8nvri3Ie73c8DeDTuIK0TqL5Y9nm0Qd1/v7le7+8FEf4jzgT+bmTUS14AG+7mE6DjsBQwmeqfR3H3KdawuJnr38paZvWRmpzUxRqZc8dRbTzRr/eQYuvtVHp1nf5ho/+plvt6YWT8zu9/MlpvZJuCejLgHAMs9zlKxXX5HMuwDDGjwul5LdFzrNTx+nQOc5x9IlEw3xDEc2SCG84H+cduvEP1DXmJmz5jZ0fHjjR3nfYjeqazM6G8q0cx9t31y94/jm5XILtraxZySZWZdgK8BZWZW/8vXCehlZocCrxO9LR9MNJOH6FRE/fM/TzS7PgF4w93rzGw90FjCzFcV0Yz9W7kauvu9RKc88uLua83sl0T/vHoT/cE3tILoD7beEKLjsDqObUy+49XLdazc/R1gnJl1AM4GHjCzPo3El6mKaIbYJHffYmYvxn0/lat5g/s3xo8d4u7rzOxMovPWEJ26GGhmlpHch5A9CVYB77v78Fzx5hlXvs4C5sXHoAp4xt1PyjqA+0vAGWZWAUwmOg05mCj2/bI8pYpoxt7X3WtaEFtL9yl1NGMP50yiUxMHAaPinwOBZ4ELPSpNfAi43sy6mtlBfDqbh+h8bg3RhbByM/sR0CNAXPcAXzazU8yszMw6x6Vsg1rSmZn93MxGmlm5mXUnOs/5rruvi2OvIzrfW+8+4AozG2pmlUSnn/4Y/+HeC5xoZl+L++tjZqPyCKPJY2VmF5jZnu5eRzSzhOi1yRZfpulAfzO73Mw6mVl3MzuykbZXAReZ2dVm1i8edxAwNI/Yq4ENZjYQ+JeMbX+L9+u78fE4m8b/8c0FNpnZD8ysS/zajmxGOeJqGj8Ou7DIQDP7MXAJ0TsDiI7XZ8xsvJlVxD9HmNmBZtYxfgfY0913El0krS/PvR34ppmdYGYd4r4PcPeVRBf+/8PMesTb9jOz40LvU9opsYfzDaLzm0vdfVX9D9Fs7Pz4LfBkoreNq4jOIf8u4/kzgMeBt4nefm+jwdv4lnD3KqIKhGuJElsVUTJp6Wvfleh0wwaii3X7EJd1xm+Nfwo8F7+VPgq4g6gKYzZRFcM2oouJuPtSorfqVxJdKJ1PI6WCDeQ6VmOBN8ysmqh65jx339ZIfJ9w981EF/S+TPQavQN8IVsA7j6H6ELiscDb8WmDJ4jO805pIvafEF1k3EhU8fNQRp87iN4FTCA63XNu5vYG49fGcY4iOq5rgd8SnZLKx43AD+Pj8P1G2gyIj2E18BLwD0QXjGfGMWwGTia6kL6C6Jj9nOidKsB44IP4lNP/IbrgjbvPJbrg+qv4ODzDp+/qLiS6GLwwPgYPkOW0YSNuAb5q0ecK/jPP56SSuevdi4hImmjGLiKSMkrsIiIpo8QuIpIySuwiIilTnDr2t2fqiq2ISHN07Q2DRuf1uZbiJPa1WlJaRKRZegyAQaPzaqpTMSIiKaPELiKSMkrsIiIpo0XARKTdqMPYUtab2vLOFL6+Xmtxymq20a32Izq0cF0zJXYRaTe2lPWmorIXlVZLowtNF5k7bPfObKmG7rXrWtSHTsWISLtRW96ZTiWc1AHMoJPVxu8qWkaJXUTaESvppF4virHlgSqxi4ikjBK7iEjCnnj2FUacein7nzKRm37zQPD+ldhFRBJUW1vLd26YyuNTf8zCv9zGfX+dzcJ3lwYdQ1UxIiJZjLngOtZu3Lrb4317dmHuPT9tcb9zF7zD/kP2Ztjg6Du/z/vS53nkf17koP2H5Hhm/pTYRUSyWLtxKwdP+tVuj78x9YqC+l2+eh2D+/f95P6g/n158bWw62fpVIyISIKyfR2pBf6wlBK7iEiCBvXvS9WqtZ/cX7ZqLQP69Q46hhK7iEiCjhg5nHeWrOD9ZavYsWMn9z/+LKd/4cigY+gcu4hIgsrLy7j1ukmc8q3rqa2r46KzTuTg4eEunIISu4hIVn17dsl6obRvzy4F933qcaM59bj8vjSjJZTYRUSyKKSksdh0jl1EJGWU2EVEUkaJXUQkZZTYRURSRoldRCRllNhFRBJ20XW30O9z4xl5+uRW6V+JXUQkYRPOOoEnpl3fav0rsYuINGHt+k18ZfL/Zd2GTcH6PHb0SHr3rAzWX0NK7CIiTbjroRmsX/4uv39wRrFDyZsSu4hII9au38T0WU/x67P3Yvqsp4LO2luTEruISCPuemgGp+1njNirM6ftZ21m1q7ELiKSRf1s/cLDewBw4eE92sysveDEbmaDzewpM3vTzN4ws++FCExEpJjqZ+t9K6O1EvtWlgebtY/7/r9z9LirWPTBcgZ94Zvc/uDMgvvMFGJ1xxrgSnefZ2bdgVfMbJa7LwzQt4hIUTw99++sWLmdPyxYucvjA9b+nX+++JyC+r7vl/9S0PNzKTixu/tKYGV8e7OZvQkMBJTYRaTNenTqDcUOocWCnmM3s32BzwIvZtk20cxeNrOXpz3yXMhhRUQkQ7Av2jCzSuBB4HJ33+3qgrtPA6YB8PyU3b+mW0Sk1TnuYFbsOJrmDtDyNBlkxm5mFURJ/V53fyhEnyIioZXVbGO7l8WJszS5w3Yvo6xmW4v7KHjGbmYG3A686e43F9qfiEhr6Vb7EVuqYVt5Z6BUp+1OWc1mutV+1OIeQpyKOQYYDywws/nxY9e6+18D9C0iEkwHnO6166C22JG0rhBVMXMo3X99IiLtjj55KtKK1m6o5itX/xfrNm4pdijSjiixi7Siux57nvWrqvj9dJX4SnKU2EVaydoN1Ux/5iV+fXZfpj/zkmbtkhgldpFWctdjz3Pa/h0Y0a8Tp+3fQbN2SYwSu0grqJ+tX3hYNwAuPKybZu2SGCV2kVZQP1vfZWVAzdolIcGWFBCRTz09721WrNnOHxas2eXxAavf5p/PP7lIUUl7ocQu0goe/Y/JxQ6hJK3dUM2km+5h2jXj6dOzW7HDSS2dihGRxKj8MxlK7CKSCJV/JkeJXUQSofLP5Cixi0irU/lnspTYRaTVqfwzWaqKEZFWp/LPZCmxi0irU/lnsnQqRqTIFi1ZzdAzr+WdqjW5G7eQlg9uX5TYRYrs6tseoHf5Vq6a8t+tNobqx9sXJXaRIlq0ZDUL3nqP353ZjQVvvdcqs3bVj7c/SuwiRXT1bQ/w9ZHlHNK/gq+PLG+VWbvqx9sfJXaRIqmfrV86pjMAl47pHHzWrvrx9kmJXaRI6mfre1eWAbB3ZVnwWbvqx9snlTuKFMmri6qYu2Mnt7+6YZfHKzpWBRtD9ePtk7l78qM+P6UIg4qItGE9BsDIr1g+TXUqRtqkEHXZSdR25zNGEvtS6Pak4kyqj7RTYpc2KURddhK13fmMkcS+FLo9qTiT6iPtlNilzQlRl51EbXc+YySxL4VuTyrOpPpoD5TYpc0JUZedRG13PmMksS+Fbk8qzqT6aA+U2KVNCVGXnURtdz5jJLEvhW5PKs6k+mgvlNilTQlRl51EbXc+YySxL4VuTyrOpPpoL1THLm1KiLrsJGq78xkjiX0pdHtScSbVR3uhOnZJ3NoN1Uy66R6mXTOePj27FTuckrdoyWrGfu8WZk65nOGD+xU7nEbl87rmahPidyO1v1+qY5dSpnK15kliWd8QQpRMqhwyDCV2SZTK1ZoniWV9QwhRMqlyyHCU2CVRKldrniSW9Q0hRMmkyiHDCZLYzewOM1tjZq+H6E/SSeVqzZPEsr4hhCiZVDlkWKFm7HcCYwP1JSmlcrXmSWJZ3xBClEyqHDKsIOWO7j7bzPYN0Zekl8rVmieJZX1DCFEyqXLIsIKVO8aJfbq7j2xk+0RgIsDUq849fOIZxwQZV0SkXSjFckd3n+buo919tJK6JGHRktUMPfPaRs9Jv7DgfXqfdAVzF37QaB/5tGltIZbUDbFsb4g4JRmqipHUylX/fekv7mbfHs6kG+9qtI982rS2JOrDVT+eLkrskkq56r9fWPA+69et444zurB+3bqsM/J82rS2JOrDVT+ePqHKHe8D/gaMMLNlZnZxiH5FWipX/felv7ib8w+pYFT/cs4/pCLrjDyfNq0tifpw1Y+nT5DE7u7j3H1vd69w90HufnuIfkVaIlf9d/1M/DtHdATgO0d03G1Gnk+b1pZEfbjqx9NJp2IkdXLVf9fPxAd0j7YP6F6224w8nzatLYn6cNWPp5OW7ZXUyVX/XbVqPbevruP2eTt32V7j6z+5nU+b1pZEfbjqx9NJy/aKiLQFpVjHLpK0JGqzc9XKh6D6cGkuJXZJrSRqs5NYK1314dJcSuySSknUZiexVrrqw6UllNgllZKozU5irXTVh0tLKLFL6iRRm53EWumqD5eWUmKX1EmiNjuJtdJVHy4tpTp2SZ0karOTWCtd9eHSUqpjFxFpC1THLsVUKnXXSaxBLlKKlNgluFKpu05iDXKRUqTELkGVSt11EmuQi5QqJXYJqlTqrpNYg1ykVCmxSzClUnedxBrkIqVMiV2CKZW66yTWIBcpZapjl2BKpe46iTXIRUqZ6thlF2s3VDPppnuYds14+vTslrXNoiWrGfu9W5g55XKGD+6XcIT5y2dfCu0jiTFEANWxS8vlUwKYxFK1ISSxbG8SY4g0lxK7fCKfEsAklqoNIYlle5MYQ6QllNjlE/mUACaxVG0ISSzbm8QYIi2hxC5AfiWASSxVG0ISy/YmMYZISymxC5BfCWASS9WGkMSyvUmMIdJSKncUIL9SxSSWqg0hiWV7kxhD2r53qtawfWdNznZzFq5kTtVOOnbq0miboUM38a8j8xtX5Y4iIs2wZet27pi5gJq6ptPYB2s2sqXPSLrtkbskuLJPf/YdeUSTbfr37MyXDx2QV7mjZuwi0u6tWb+Zf/vveZRVdMzZdu3mHRx4+rfp3K2yyXbDKjrRtbJ7qBCbRYldRNqcZWvWs21H06c41mzYwpRZ79GtZ++c/dVYBYef92M6d03HB8SU2EWkJDz83Fus3vhxznZL12xiedkguu85oMl2VtaL4y6dRFl5+0tz7W+PRSQRdXV13PjAXNZtqcvZdvPH2+h+yMn0O/QfcrbtU9GRob37hggxtZTYRQSATVu2smnLtpztNlZv4+d/WUinHk2f4ti5s4ahx4/jwGEHhQpR8qTELpJyT857j+Xrqptss31nLU++XU3fYbnr6bxDN4685Gd07NQ5VIgSmBK7SBvj7kz5yzyWrNuas+2WrTsoH3YU/UeMztn2hBOG0KGsLESIUmRK7CKtbNv2nVRv3d5kG3fnFw+/yiaaLqEDqKmtZe/RYzng1KNChSgpo8Qu0gLvLvuQee/lXiOnpraWP8//kF5DDszZdp8vTuIzg4aFCE/auSCJ3czGArcAZcBv3f2mEP2KJO3OJxfwetWGJtu4w5raSoYd+9W8+vzC0cMoz+ODLyKhFJzYzawMuA04CVgGvGRmj7r7wkL7FmmMu7Nl64682t386Kus2pY7sdbW1tHrwGMYMf6knG0PyCtKkeIIMWMfA7zr7osBzOx+4AxAiV2abcWHG3nlnRU5202ft4ya3vuB5V46Y8ioczlseO76aJG0CJHYBwKZy/stA45s2MjMJgITAaZedS4TzzgmwNDSFrz89gr+OOcdLI8kvOzjCvY99pyc7T5z7kB67KEPqYhkEyKxZ/tr3W3ZM3efBkwDtLpjiaupqc3Z5olXFvPI/A/p2LEiZ9sdnftwxPif55XYh+cVoYg0JURiXwYMzrg/CMj9XloSs2NnDc+9tpi6PJZonj5vOesq9qJDh6brmXsN3I8xF18eKkQRCShEYn8JGG5mQ4HlwHnA1wP0KzksqvqQO57MfSmjat3H9PvHc6hoYhH/env90yAO2qvpxZVEpLQVnNjdvcbMJgMziMod73D3NwqOrJ2a8/pSfj9nCZ3y+Lj2Frpw5Lif5Vy9bv9QwYlImxCkjt3d/wr8NURfbUldXR0vLlxCXY5vUnmj6iNmvbedLpU9cvZZ2XcgR1x8Y17no0VEstEnTxvYtn0ntzzyMh/vyH0BcfHqTfT87D/RsUvTHwPvMqQnJ59yaKgQRUSa1C4S+/pNH/Ov972Ideyas+2GLds56IzL2KvPnjnb7t2hrF0u4i8ipa0ks9K7yz5kY3XuleveWbmRB+d/RLeeezTZrs7KOPxrP6Jr99ynQkRE2rqiJPYb//RCo9vWb97K6s77sMfgETn76ditGydfpg86iYhkKkpi73ryVY1uq+zQgeEp+UJZEZFiKE5ir+xejGGlQDdOHkd19ebdHq+s7M41t95XhIhEJJuSPMcupam6ejPDLpmy2+OLf3tZEaIRkcZ0KHYAIiISlhK7iEjKKLGLiKSMEruISMro4qnkrbKye9YLpZWqchIpKUrskjeVNIq0DUrsEozq3EVKgxK7BKM6d5HSoIunIiIpo8QuIpIySuwiIimjxC4ikjK6eCrBqM5dpDQosUvecpUzrly6mJosX+y9+aMPg8Vw6Smj8LKy3R632lp+PWN+sHFUuiltmRK75C1XOWNNnbPP5Lt2277k1guDxeBlZQyefM9uj1fdekGwMUClm9K26Ry7iEjKKLGLiKSMEruISMoosYuIpIwunkrecpUzlnewrBdKyztYsBistjbrhVKrrQ02Bqh0U9o2c9+9PK21/Wb24uQHFRFpw/r37MyXDx2Q1yxJM/YSkKtmOkRNdRJ9fPf0I7PWsZd3MP7z0RfzGqNUqI5d2jIl9hKQq2Y6RE11En0kUceeFNWxS1umi6ciIimjxC4ikjJK7CIiKaPELiKSMgVdPDWzc4DrgQOBMe7+coig2ptcNdMhaqqT6COJOvakqI5d2rKC6tjN7ECgDpgKfD/fxK46dhGR5kmsjt3d3wQwa3szsqSEWD88Vx/51I/n6iOfuu1JJx+ClVXs1sZrdzJ15mtB4gxRK59EPb3q3KWUJVbHbmYTgYkAF1x5A8eePi6poYsqxPrhufrIp348Vx/51G1bWQWDL8vSx5QLgsUZolY+iXp61blLKcuZ2M3sSaB/lk3Xufsj+Q7k7tOAaaBTMSIirSlnYnf3E5MIREREwlC5o4hIyhSU2M3sLDNbBhwNPGZmM8KEJSIiLVVoVczDwMOBYkmlEOuH5+ojn/rxXH3kU7fttTs/uVCayWt3BoszRK18EvX0qnOXUqb12AvUVsrecsUZoiyz0BhEpHFajz1BbaXsLVecIcoyC41BRMLQxVMRkZRRYhcRSRkldhGRlFFiFxFJGV08LVBbKXvLFWeIssxCYxCRMFTuKCLSBjSn3FGnYkREUkaJXUQkZZTYRURSRoldRCRllNhFRFJGiV1EJGWU2EVEUkaJXUQkZZTYRURSRoldRCRllNhFRFJGiV1EJGWU2EVEUkaJXUQkZZTYRURSRoldRCRllNhFRFJGiV1EJGWU2EVEUkaJXUQkZZTYRURSRoldRCRllNhFRFJGiV1EJGWU2EVEUkaJXUQkZQpK7Gb272b2lpm9ZmYPm1mvUIGJiEjLFDpjnwWMdPdDgLeBawoPSUREClFQYnf3me5eE999ARhUeEgiIlKIkOfYLwIeb2yjmU00s5fN7OXZj94XcFgREclUnquBmT0J9M+y6Tp3fyRucx1QA9zbWD/uPg2YBvCb2Yu9RdGKiEhOORO7u5/Y1HYz+wZwGnCCuythi4gUWc7E3hQzGwv8ADjO3T8OE5KIiBSi0HPstwLdgVlmNt/M/itATCIiUoCCZuzuvn+oQEREJAx98lREJGWU2EVEUkaJXUQkZQo6x95Sfbt3LMawIiJtVq+uFXm3NZWe58fMJsYfskqVNO5XGvcJtF9tTTH3S6di8jex2AG0kjTuVxr3CbRfbU3R9kuJXUQkZZTYRURSRok9f6k7BxhL436lcZ9A+9XWFG2/dPFURCRlNGMXEUkZJXYRkZRRYm+GNH55t5mdY2ZvmFmdmY0udjyFMrOxZrbIzN41s6uLHU8IZnaHma0xs9eLHUsoZjbYzJ4yszfj37/vFTumEMyss5nNNbO/x/v1k2LEocTePGn88u7XgbOB2cUOpFBmVgbcBnwJOAgYZ2YHFTeqIO4ExhY7iMBqgCvd/UDgKOA7KXmttgNfdPdDgVHAWDM7KukglNibIY1f3u3ub7r7omLHEcgY4F13X+zuO4D7gTOKHFPB3H028FGx4wjJ3Ve6+7z49mbgTWBgcaMqnEeq47sV8U/iFSpK7C3X5Jd3S1EMBKoy7i8jBcki7cxsX+CzwIvFjSQMMyszs/nAGmCWuye+X0VZBKyUhfry7lKSzz6lhGV5TPW8JczMKoEHgcvdfVOx4wnB3WuBUfE1uIfNbKS7J3p9RIm9gTR+eXeufUqRZcDgjPuDgBVFikVyMLMKoqR+r7s/VOx4QnP3DWb2NNH1kUQTu07FNEPGl3efri/vLkkvAcPNbKiZdQTOAx4tckyShZkZcDvwprvfXOx4QjGzPeur5cysC3Ai8FbScSixN0/qvrzbzM4ys2XA0cBjZjaj2DG1VHxhezIwg+hi3J/c/Y3iRlU4M7sP+BswwsyWmdnFxY4pgGOA8cAX47+l+WZ2arGDCmBv4Ckze41oojHL3acnHYSWFBARSRnN2EVEUkaJXUQkZZTYRURSRoldRCRllNhFRFJGiV1EJGWU2EVEUuZ/ATqvoN2ETblaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4XPV97/H3d0a7LGlkW5KtxRiC2WwjyREQSktDSWggrokJlyzQrC2lTZM097a5oX2e/NHn6b25N12SJvdCgeyhae8lEAhbIFtJegvBm7wSQsAgWbYlL1osWfv3/jFHgyxrNT5zZjSf1/PMM2fOOZr5YOzz0Tlnzu+YuyMiIgIQizqAiIhkDpWCiIikqBRERCRFpSAiIikqBRERSVEpiIhIikpBRERSVAoiIpKiUhARkZS8qAMs1PLly3316tVRxxARySpbt2494u5Vc62XdaWwevVqtmzZEnUMEZGsYmavzmc9HT4SEZEUlYKIiKSoFEREJEWlICIiKSoFERFJUSmIiEiKSkFERFJyphR+eaiPv3lsLwPDo1FHERHJWDlTCu3HB7j3Z6+wq70n6igiIhkrZ0qhsSEBwI627oiTiIhkrpwpheVLCqmvLKa1XaUgIjKTUEvBzBJm9oCZvWBm+8zsyinL32pmPWa2I3h8Nsw8TQ0JdrymUhARmUnYA+J9EXjS3W82swKgZJp1fubuG0POASRL4dGdB+nsHaS6vCgdHykiklVC21Mws3LgauArAO4+7O6R/prevCp5XmG7ziuIiEwrzMNH5wFdwNfMbLuZ3WdmpdOsd6WZtZrZE2a2NsQ8rK2tIC9mtKoURESmFWYp5AEbgLvcvRnoBz4zZZ1twDnu3gh8CfjedG9kZreb2RYz29LV1XXGgYry41y0skzfQBIRmUGYpdAOtLv7c8HrB0iWRIq797r7iWD6cSDfzJZPfSN3v8fdW9y9papqzhsHzaqpIcHO9h7Gxv0NvY+IyGIUWim4+yGgzcwuDGZdC+ydvI6ZrTAzC6YvD/IcDSsTQFNDJSeGRvl114kwP0ZEJCuF/e2jjwP3B988ehn4sJndAeDudwM3A39sZqPASeC97h7qr/BNDRVA8iK2C2rKwvwoEZGsE2opuPsOoGXK7LsnLf8y8OUwM0x13vIllBXlsaOtm1taGtL50SIiGS9nrmieEIsZjfW6iE1EZDo5VwqQPNn8y8N9nBweizqKiEhGyclSaGxIMDbu7DqgEVNFRCbLyVJoCkZM1UVsIiKnyslSqCorpC5RrIvYRESmyMlSAGhalVApiIhMkbOl0NyQ4ED3STr7BqOOIiKSMXK2FBpT5xV0sllEZELOlsK62griMWNH2/Goo4iIZIycLYXigjgXrdCIqSIik+VsKUAwYmpbD+MaMVVEBMjxUmhsSNA3NMrLRzRiqogI5HgpNAcnm7drHCQRESDHS+FNVUsoK8zTeQURkUBOl0IsZlzaUKFSEBEJ5HQpQPJk8wuHNGKqiAioFGisT46YuqdDF7GJiOR8KTStSp5s1iEkERGVAtVlRdQlitmuUhARUSlA8ryCbs8pIqJSAKCxoYID3Sfp6huKOoqISKRUCkBTQyWgO7GJiKgUgPV1EyOmqhREJLepFEiOmHphjUZMFRFRKQQaGxK0tndrxFQRyWkqhUBzQ4K+wVFePtIfdRQRkcioFAK6iE1EJORSMLOEmT1gZi+Y2T4zu3LKcjOzfzSzl8xsp5ltCDPPbN5UtYQlhXm6PaeI5LS8kN//i8CT7n6zmRUAJVOWXw+sCR5XAHcFz2kXjxmX1mvEVBHJbaHtKZhZOXA18BUAdx9296lb3BuBb3rSs0DCzFaGlWkujQ0JXjjYx+CIRkwVkdwU5uGj84Au4Gtmtt3M7jOz0inr1AFtk163B/NOYWa3m9kWM9vS1dUVWuCmhgSjGjFVRHJYmKWQB2wA7nL3ZqAf+MyUdWyanzvtO6Hufo+7t7h7S1VV1dlPGtDtOUUk14VZCu1Au7s/F7x+gGRJTF2nYdLreqAjxEyzqi4voraiSOcVRCRnhVYK7n4IaDOzC4NZ1wJ7p6z2CPCB4FtIbwF63P1gWJnmY+IiNhGRXBT2t48+DtwffPPoZeDDZnYHgLvfDTwO3AC8BAwAHw45z5yaGhI8sfsQR08MsWxJYdRxRETSKtRScPcdQMuU2XdPWu7Ax8LMsFBNDa9fxHbtxTURpxERSS9d0TzF+nqNmCoiuUulMEVJQR4XaMRUEclRKoVpNDVU0NqmEVNFJPeoFKbR1JCgd3CUV45qxFQRyS0qhWlM3J5zhy5iE5Eco1KYxvnVSygtiOu8gojkHJXCNOIxY319hS5iE5Gco1KYQVNDJfsO9mrEVBHJKSqFGTQ1JBgZc/Z09EYdRUQkbVQKM2jW7TlFJAepFGZQU17EivIiWlUKIpJDVAqzaGpIaE9BRHKKSmEWTasSvHZsgKMnhqKOIiKSFiqFWUyMmKqvpopIrlApzGJ9XQUx05XNIpI7VAqzKC0MRkxt74k6iohIWqgU5tDUkKC1rZvk/YBERBY3lcIcmhoS9Jwc4ZUjGjFVRBY/lcIcmnQRm4jkEJXCHNZUl1FSENdFbCKSE1QKc4jHjPV1FdpTEJGcoFKYh6ZVCfZqxFQRyQEqhXloDkZM3XtQI6aKyOKmUpiHxokrm3UISUQWOZXCPKysKKamvFDnFURk0VMpzJNGTBWRXBBqKZjZfjPbZWY7zGzLNMvfamY9wfIdZvbZMPO8EU0Nlbx6dIBj/cNRRxERCU1eGj7jGnc/Msvyn7n7xjTkeEOaJp1XuOai6ojTiIiEQ4eP5ml9fQVmurJZRBa3sEvBgafMbKuZ3T7DOleaWauZPWFma0POc8aWFOZxQXWZSkFEFrWwDx9d5e4dZlYNPG1mL7j7M5OWbwPOcfcTZnYD8D1gzdQ3CQrldoBVq1aFHHlmTQ0JfrD3EO6OmUWWQ0QkLKHuKbh7R/DcCTwEXD5lea+7nwimHwfyzWz5NO9zj7u3uHtLVVVVmJFn1bQqQffACPuPDkSWQUQkTKGVgpmVmlnZxDRwHbB7yjorLPiV28wuD/IcDSvTG9VYr4vYRGRxC3NPoQb4uZm1Ar8AHnP3J83sDjO7I1jnZmB3sM4/Au/1DL6bzQU1SyjOj+u8gogsWqGdU3D3l4HGaebfPWn6y8CXw8pwtuXFY6yvr2C7SkFEFil9JXWBmhsS7OvoZWhUI6aKyOKjUligpoYEw2Pj7DvYF3UUEZGzTqWwQBMjpu547XjESUREzj6VwgKtrCiiukwjporI4qRSWCAz04ipIrJoqRTOQNOqBPuPDnBcI6aKyCKjUjgDTRMXsbVrb0FEFpd5lYKZfWs+83KFRkwVkcVqvnsKp4xeamZx4M1nP052KCvKZ031EpWCiCw6s5aCmd1pZn3ApWbWGzz6gE7g4bQkzFBNDQla27rJ4FE5REQWbNZScPf/7u5lwOfdvTx4lLn7Mne/M00ZM1JjQ4LjAyO8dkwjporI4jHfw0ePBiOdYma3mdnfm9k5IebKeBO359QhJBFZTOZbCncBA2bWCHwaeBX4ZmipssCFNWUU58fZ/ppKQUQWj/mWwmgwpPWNwBfd/YtAWXixMl9ePMb6ugrtKYjIojLfUugzszuB3wceC759lB9erOzQtCrB3o5ehkfHo44iInJWzLcU3gMMAR9x90NAHfD50FJlicb6iRFTe6OOIiJyVsyrFIIiuB+oMLONwKC75/Q5BUjuKYBONovI4jHfK5pvIXlLzf8E3AI8Z2Y3hxksG9RWFFGlEVNFZBGZ7+04/wq4zN07AcysCvgh8EBYwbLBxIip2147jrtjZlFHEhF5Q+Z7TiE2UQiBowv42UXt6guqePXoAHs6dF5BRLLffDfsT5rZD8zsQ2b2IeAx4PHwYmWPjetXkh83Htp+IOooIiJv2FxjH51vZle5+18A/wRcCjQC/wHck4Z8Ga+ytIDfuaiah3d0MDqmr6aKSHaba0/hC0AfgLs/6O7/2d0/RXIv4Qthh8sWm5vrOXJiiJ+9dCTqKCIib8hcpbDa3XdOnenuW4DVoSTKQtdcVEWiJJ8Ht+kQkohkt7lKoWiWZcVnM0g2K8yLs/HSlTy15xB9gyNRxxEROWNzlcLzZvaHU2ea2UeBreFEyk6bm+sZGh3nid2Hoo4iInLG5rpO4c+Ah8zsVl4vgRagANg815ub2X6S5yTGSA6q1zJluQFfBG4ABoAPufu2hfwHZIoNqxKsXlbCg9vauaWlIeo4IiJnZNZScPfDwG+Y2TXAumD2Y+7+4wV8xjXuPtMZ2OuBNcHjCpJDdF+xgPfOGGbG5uZ6/uGHL3Kg+yR1CR1dE5HsM9+xj37i7l8KHgsphLncCHzTk54FEma28iy+f1ptbq4D4Hu6ZkFEslTYVyU78JSZbTWz26dZXge0TXrdHszLSquWlXDZ6koe3NauezeLSFYKuxSucvcNJA8TfczMrp6yfLrBgk7bmprZ7Wa2xcy2dHV1hZHzrNncXM+vu/rZdaAn6igiIgsWaim4e0fw3Ak8BFw+ZZV2YPJZ2XqgY5r3ucfdW9y9paqqKqy4Z8U716+kIC+maxZEJCuFVgpmVmpmZRPTwHXA7imrPQJ8wJLeAvS4+8GwMqVDRUk+b7u4mu+3djCiYS9EJMuEuadQA/zczFpJ3ovhMXd/0szuMLM7gnUeB14GXgLuBf4kxDxpc1NzPUf7h3nmxcw+1CUiMtV876ewYO7+MsnB86bOv3vStAMfCytDVH77wiqWlhbw4LYDXHtxTdRxRETmTfdECEF+PMbvXbqSp/cdpuekhr0QkeyhUgjJTRvqGR4d5/FdWX2KRERyjEohJJfWV3BeVSkP6VtIIpJFVAohMTPevaGeX+w/RtuxgajjiIjMi0ohRDc21QLoVp0ikjVUCiGqryzhinOX8tD2Axr2QkSygkohZO/eUM8rR/rZ3tYddRQRkTmpFEJ2/foVFObFdMJZRLKCSiFkZUX5XLd2Bd/f2cHwqIa9EJHMplJIg5ua6+geGOEnv+yMOoqIyKxUCmnwW2uWs3xJgQ4hiUjGUymkQV48xqbGOn78QifdA8NRxxERmZFKIU1u2lDH8Ng4j+7UsBcikrlUCmmytracC2qW6EI2EcloKoU0MTM2N9ez9dXj7D/SH3UcEZFpqRTS6F3NtZhp2AsRyVwqhTRaWVHMb7xpmYa9EJGMpVJIs83N9bx2bICtrx6POoqIyGlUCmn2jnUrKM6P86AOIYlIBlIppNmSwjx+d20Nj7Z2MDgyFnUcEZFTqBQicNOGenoHR/nJCxr2QkQyi0ohAledv5zqskIdQhKRjKNSiEA8ZtzYVMtPXujkWL+GvRCRzKFSiMhNG+oZHXce3dkRdRQRkRSVQkQuXlnORSvKeFAjp4pIBlEpROjdG+rZ0dbNr7tORB1FRARIQymYWdzMtpvZo9Ms+5CZdZnZjuDxB2HnySQ3NtUSM/ieTjiLSIZIx57CJ4F9syz/V3dvCh73pSFPxqguL+Kq85fz4LYDjI9r2AsRiV6opWBm9cA7gZza2C/EuzfUc6D7JM/vPxZ1FBGR0PcUvgB8GpjtjvXvNrOdZvaAmTWEnCfjXLe2hpKCuEZOFZGMEFopmNlGoNPdt86y2veB1e5+KfBD4BszvNftZrbFzLZ0dXWFkDY6JQV5XL9uJY/tPKhhL0QkcmHuKVwFbDKz/cC/AL9jZt+evIK7H3X3oeDlvcCbp3sjd7/H3VvcvaWqqirEyNG4aUMdfUOj/HDf4aijiEiOC60U3P1Od69399XAe4Efu/ttk9cxs5WTXm5i9hPSi9ZbzlvGivIiXbMgIpFL+3UKZvbXZrYpePkJM9tjZq3AJ4APpTtPJojHjHc11/FvL3Zx5MTQ3D8gIhKStJSCu//U3TcG059190eC6Tvdfa27N7r7Ne7+QjryZKKbNtQxNu58v1XDXohIdHRFc4a4oKaMdXXlOoQkIpFSKWSQzc317DrQw68O90UdRURylEohg2xqrCUeM91nQUQio1LIIFVlhVy9ZjkPb9ewFyISDZVChtm8oZ6OnkGefeVo1FFEJAepFDLMdZfUUFaYpxPOIhIJlUKGKcqPc/36FTyx6yAnhzXshYikl0ohA21urqd/eIyn9h6KOoqI5BiVQga64tylNCwt5n8++Uvajg1EHUdEcohKIQPFYsZdt76ZvsER3n/fsxzoPhl1JBHJESqFDLWuroJv/8EVdA+M8P57n+VQz2DUkUQkB6gUMtil9Qm++ZHLOXpimPfd+yydvSoGEQmXSiHDNa+q5BsfuYzO3sFkMfSpGEQkPCqFLPDmc5bytQ9fTkf3ILfe+5yG1xaR0KgUssTl5y7lqx+6jLbjA9x233Mc6x+OOpKILEIqhSxy5ZuW8ZUPXsYrR/q57b7n6B5QMYjI2aVSyDJXnb+cez/QwktdJ7jtK8/RMzASdSQRWURUClno6guq+Kfb3syLh07wga8+R++gikFEzg6VQpa65qJq/vetG9h7sJcPfvUX9KkYROQsUClksbddUsOX3reBXe09fPhrz9M/NBp1JBHJciqFLPeOdSv4x/c1s72tmw9//XkGhlUMInLmVAqLwA3rV/IP72liy/5jfPTrWzTktoicMZXCIrGpsZa/u6WRZ185yu3f2sLgiIpBRBZOpbCIbG6u5/M3N/Lzl47wR9/aytCoikFEFkalsMjc/OZ6PnfTev7txS7++NvbGB4djzqSiGQRlcIi9J7LVvE3m9fx4xc6+dg/b2NkTMUgIvOjUlikbr3iHP76xrU8vfcwn/jOdhWDiMxL6KVgZnEz225mj06zrNDM/tXMXjKz58xsddh5cskHrlzNZzdewhO7D/Gpf93BqIpBROaQl4bP+CSwDyifZtlHgePufr6ZvRf4H8B70pApZ3zkN89lbNz5m8f3EY8Zf39LE/GYRR1LRDJUqHsKZlYPvBO4b4ZVbgS+EUw/AFxrZtpinWV/ePV5fPodF/Lwjg7+4v+2ao9BRGYU9p7CF4BPA2UzLK8D2gDcfdTMeoBlwJHJK5nZ7cDtAKtWrQot7GL2J289n7Ex5++efpFnftXFO9evZFNTLRtWVaIeFpEJoZWCmW0EOt19q5m9dabVppnnp81wvwe4B6ClpeW05TI/H792DZfUlvPdbe38y/NtfOM/XqUuUcymplo2NdZy0YoyFYRIjgtzT+EqYJOZ3QAUAeVm9m13v23SOu1AA9BuZnlABXAsxEw579qLa7j24hr6Bkd4as9hHmnt4J5nXuaun/6aNdVL2NRYy6amWs5ZVhp1VBGJgLmH/4t3sKfw5+6+ccr8jwHr3f2O4ETzTe5+y2zv1dLS4lu2bAkvbA46emKIx3cd5JHWDp7ffxyAxoYEmxpr+b1LV1JdXhRxQhF5o8xsq7u3zLleukvBzP4a2OLuj5hZEfAtoJnkHsJ73f3l2d5LpRCuA90nebS1g4d3dLD3YC8xg7ect4xNjbVcv24lFSX5UUcUkTOQUaVwNqkU0uelzhM80trB91s7eOVIP/lx47cvqGZTUy1vu7iakoJ0fKNZRM4GlYKcNe7O7gO9PLzjAI/uPMih3kFKCuK8/ZIaNjXW8ltrqijI08XxIplMpSChGB93frH/GA/v6OCJ3QfpHhghUZLPdZfUsGFVJevqKlhTs4TCvHjUUUVkEpWChG54dJyfv9TFIzs6+NG+TvqC24Hmx40LaspYV1vBurpy1tZVcPGKcooLVBQiUVEpSFqNjzuvHRtgd0cPuw/0sqejh10HeugeGAEgZnB+9RLW1VUEZVHBJbXlLCnUeQmRdFApSOTcnY6eQXYf6GHPgR52d/Sy+0APnX1DqXXOW17K2roK1tWWs66ugrW15SRKCiJMfTp3xz15VaW748B4MC+5PPk6Zqa9IclY8y0F/ZomoTEz6hLF1CWK+d21K1LzO3sH2RMUxO6OHra9epzvt3akltdXFqcOPdVVFjMy6gyNjTM8mnwMjY6lpocnz580PXXZxPTQ6DjDo2OMjXuwkQfHGXcgmJ7YyE8sX4iyojzqK0toqCymYWnyub6yJDm9tFjf2Fok3J3ek6N09g3S2TfE4d5Tn0fHxkkUF5AozaeypIDKknwSJQUkivOpLC0gUZJPorggI7+gob+hknbV5UVUlxdxzUXVqXnH+ofZExx62t2R3LN4cs+hGd8jL2YU5MWSj3jslOnCYLooP0Z5UV6wLJ5arzAvRjxmGGAGMTMwMCx4/fp0cp2J6UnLg+FAJn7egDF3DvcM0nb8JPuP9vOzXx3h5JR7ZS8rLaC+spj6pSU0VCaLYqJE6iqLM/4EvbtzYmiUY/3DHDkxzLH+YY6eGOJo/zBHTwxzfGCYmBllRXmUF+VRVpRP2SnPyemJZUX5sYwaWsXd6R4YOWUD39k3SGdv8vlw7+uvh6a5q2FpQZzq8iLyYkb3yW66B4YZGZv5N4vSgjiJkgIqg/KoKJ5SIiXJ1xPPVWWFlIZ8yFWHjyRj9Q6OcPTE8Ckb/sK8GPnxWFYM/+3uHO0fpu3YAG3HT9J+fIC2YxPPAxzoPnnKBsMMasqKTimK+qUlrKwooiAeIy8eIz9u5MWC53iMvJiRH4+RFzfyY8nnienYPP+MBoZHOXpiONiwv76BP3piKLnx7x/mWP9Qap2ZbvFaWhCnsrQA9+T/uxNDo3PuaeXF7LTSKC/KT72eKI8lRXkYMD6xF+eemh53gtc+aXnyPNdc6w+OjNPVN8ThYEPf1TfE8DSjCJcV5lFdXkh1WRHV5YXUlBdRXVZIVdnr09XlRaedI3N3+ofH6B4YpntghOPBc/fAMMeD1z3B8/GBEXpOBvNOjkz7Z3f71efxlzdcPK//r1PpnIJIhhsbdzr7Bmk7djIojgHajyen24+f5GDPyeRhrTMUM5JFErNTCiUvniyS4dFxjvYPMTgy/Ua+KD/GstJCli0pYFlpAUtLC1m+pIClpQUsW1LIstKC5LJguij/1L2c8XGnf3iUvsGJxwi9gyP0DY7SG7zuO+X51On5Fst8TOzRTezlxYLX+fFYsEEvpKasiKpgw18zUQDBsnQf9hsbd3pPjtB9cqJIhjneP8L51UtobEic0XvqnIJIhovHjJUVxaysKObyc5eetnxkbJyD3YMc6h1kZGyckbFxRsec0fFxRiY/T5pOrvP68tExn7LuOKPjyfUK4rHXN/DBhn/yxv6NbghjMQt+4z/zoVEmiuVE8HXnmNmkDfzpG/lTl7++fiYdopqPeMyoLC2gsrSAc0nv4JQqBZEMlR+PsWpZCauWlUQdJTJno1hkYTLv1LeIiERGpSAiIikqBRERSVEpiIhIikpBRERSVAoiIpKiUhARkRSVgoiIpGTdMBdm1gW8eoY/vhw4chbjhC2b8mZTVsiuvNmUFbIrbzZlhTeW9xx3r5prpawrhTfCzLbMZ+yPTJFNebMpK2RX3mzKCtmVN5uyQnry6vCRiIikqBRERCQl10rhnqgDLFA25c2mrJBdebMpK2RX3mzKCmnIm1PnFEREZHa5tqcgIiKzyJlSMLN3mNkvzewlM/tM1HlmYmYNZvYTM9tnZnvM7JNRZ5oPM4ub2XYzezTqLLMxs4SZPWBmLwR/xldGnWk2Zvap4O/BbjP7jpkVRZ1pMjP7qpl1mtnuSfOWmtnTZvar4LkyyowTZsj6+eDvwk4ze8jMzuy2ZiGYLu+kZX9uZm5my8/25+ZEKZhZHPhfwPXAJcD7zOySaFPNaBT4L+5+MfAW4GMZnHWyTwL7og4xD18EnnT3i4BGMjizmdUBnwBa3H0dEAfeG22q03wdeMeUeZ8BfuTua4AfBa8zwdc5PevTwDp3vxR4Ebgz3aFm8XVOz4uZNQBvB14L40NzohSAy4GX3P1ldx8G/gW4MeJM03L3g+6+LZjuI7nRqos21ezMrB54J3Bf1FlmY2blwNXAVwDcfdjdu6NNNac8oNjM8oASoCPiPKdw92eAY1Nm3wh8I5j+BvCutIaawXRZ3f0pdx8NXj4L1Kc92Axm+LMF+Afg00AoJ4RzpRTqgLZJr9vJ8A0tgJmtBpqB56JNMqcvkPxLOv0d4DPHeUAX8LXgUNd9ZpbeG+AugLsfAP6W5G+EB4Eed38q2lTzUuPuByH5Sw5QHXGe+foI8ETUIWZjZpuAA+7eGtZn5EopTHfX7oz+2pWZLQG+C/yZu/dGnWcmZrYR6HT3rVFnmYc8YANwl7s3A/1kzqGN0wTH4m8EzgVqgVIzuy3aVIuTmf0VyUO390edZSZmVgL8FfDZMD8nV0qhHWiY9LqeDNsNn8zM8kkWwv3u/mDUeeZwFbDJzPaTPCz3O2b27WgjzagdaHf3iT2vB0iWRKZ6G/CKu3e5+wjwIPAbEWeaj8NmthIgeO6MOM+szOyDwEbgVs/s7+i/ieQvCK3Bv7d6YJuZrTibH5IrpfA8sMbMzjWzApIn6x6JONO0zMxIHvPe5+5/H3Weubj7ne5e7+6rSf65/tjdM/K3WXc/BLSZ2YXBrGuBvRFGmstrwFvMrCT4e3EtGXxifJJHgA8G0x8EHo4wy6zM7B3AfwU2uftA1Hlm4+673L3a3VcH/97agQ3B3+uzJidKITiR9KfAD0j+o/o/7r4n2lQzugr4fZK/ce8IHjdEHWoR+Thwv5ntBJqA/xZxnhkFezQPANuAXST/vWbUFbhm9h3gP4ALzazdzD4KfA54u5n9iuS3ZD4XZcYJM2T9MlAGPB38W7s70pCTzJA3/M/N7L0lERFJp5zYUxARkflRKYiISIpKQUREUlQKIiKSolIQEZEUlYLkHDM7ETyvNrP3n+X3/sspr//f2Xx/kbCpFCSXrQYWVArBiLuzOaUU3D0brkAWSVEpSC77HPBbwUVLnwruCfF5M3s+GF//jwDM7K3BPS7+meRFZJjZ98xsa3Cvg9uDeZ8jOaLpDjO7P5g3sVdiwXvvNrNdZvaeSe/900n3eLg/uHoZM/ucme0Nsvxt2v90JCflRR1AJEKfAf7c3TcCBBv3Hne/zMwKgX83s4lRSS8nOe7+K8Hrj7j7MTMrBp43s++6+2fM7E/dvWmaz7o0rXlOAAABfUlEQVSJ5BXUjcDy4GeeCZY1A2tJjsf178BVZrYX2Axc5O6eSTd/kcVNewoir7sO+ICZ7SA5XPkyYE2w7BeTCgHgE2bWSnIM/oZJ683kN4HvuPuYux8G/g24bNJ7t7v7OLCD5GGtXmAQuM/MbgIyelweWTxUCiKvM+Dj7t4UPM6ddP+C/tRKZm8lOYLple7eCGwH5rpN5nTDt08YmjQ9BuQF43VdTnK03HcBTy7ov0TkDKkUJJf1kRwMbcIPgD8Ohi7HzC6Y4SY8FcBxdx8ws4tI3jZ1wsjEz0/xDPCe4LxFFck7wP1ipmDB/TQq3P1x4M9IHnoSCZ3OKUgu2wmMBoeBvk7y/s2rSY5RbyTv0jbdrSSfBO4IRlr9JclDSBPuAXaa2TZ3v3XS/IeAK4FWkjd4+rS7HwpKZTplwMNmVkRyL+NTZ/afKLIwGiVVRERSdPhIRERSVAoiIpKiUhARkRSVgoiIpKgUREQkRaUgIiIpKgUREUlRKYiISMr/ByI6Dll+kVi1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "from mlxtend.classifier import Adaline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "X = X[0:100] # class 0 and class 1\n",
    "y = y[0:100] # class 0 and class 1\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "\n",
    "ada = Adaline(epochs=15, \n",
    "              eta=0.02, \n",
    "              minibatches=len(y), # for SGD learning \n",
    "              random_seed=1,\n",
    "              print_progress=3)\n",
    "\n",
    "ada.fit(X, y)\n",
    "plot_decision_regions(X, y, clf=ada)\n",
    "plt.title('Adaline - Stochastic Gradient Descent')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(ada.cost_)), ada.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 4 - Stochastic Gradient Descent with Minibatches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Iteration: 15/15 | Cost 3.87 | Elapsed: 0:00:00 | ETA: 0:00:00"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEICAYAAABLdt/UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmYXGWZ9/HvL93ZSIewBAjZ2GXLQEQIICKMgEQGQURElgCDGgbFBVFUmFfxfWFgHEeHAd+RKIgsAg6LYhRZHBaRXfZ9CyEhCyRkJZCkO/f8cU5D0emqru46far69O9zXblSVefU89znnKq7nnrO3acUEZiZWXEMqHcAZmaWLSd2M7OCcWI3MysYJ3Yzs4JxYjczKxgndjOzgunziV3SCZLurnLdsyRdkd4eL2m5pKbejbB+JF0q6ewc+jlG0i293U9Ple4HSXtLeq7eMVl1JB0v6aburiupWVJI2rw340v7ukLSWb3dT3c0bGKXdIekRZIG90b7EfFqRLRERFtvtF+JpLGSrpO0QNISSU9IOiFdtnn6gmzOO65qdBZfRFwZER/vYXu7SpqeHuvFkp6WdI6k9bOL+j0R8ZeI2DaLtiS9Imn/Csv3lbQmHUAslzRb0m8k7ZZF/70hfd99IYd+rkhfRwd1ePzC9PFjASLiVxHxiWra7M66VcQ3W9K+WbRVDw2Z2NNP2b2BAA6pazC943JgFrAZsCFwHDC/rhHVgaQPA3cAfwW2i4j1gMlAK7Bzmec05AdeBXMiogUYDuwBPAv8RdJ+9Q2rITwPHN9+R9JA4HDg5bpFVBANmdhJEt19wKWUHHgASRtKulHSUkkPAFt1WH6+pFnp8r9J2ruzDjqOPNORyv+T9FdJyyTdImlkyfp7SLonHVU+VuOn+W7ApRHxVkS0RsQjEdH+dfOu9P/F6ShvT0kDJP2zpJmSXpd0maQRJbF9pCS2We2j/9T6kv6QbtP9krYqeV7ZfSVpkqSH0mXzJf24Qnzvmw6TtKOkWyW9mT73jDL74YfALyPi3IiYD+9+k/p+RNyRtnVCekx+IulN4CxJW0n6H0kL0289V0par6T/D0p6ON3ma4AhJcv2lTS75P7o9NvTG5JmSPpqybKz0hH2ZWlbT0naNV12OTAe+H26H04vs42k2xURMTsivgf8AvjXkn62K9lfz0n6bMmyg9JvMcskvSbpmyXLDpX0aHqMXpI0OX18hKSLJc1Nn3O20inH9mMl6UdKviXNkPSJdNk5JAOqC9NturDjdkj6laTT0ttj0vfQl9L7W6fboEr7osRvgX1LXsv/ADwEvFHS3xck3ZHebp9eOUnSi2n8/9nZuiU+mW7jAknnSRqQrruNpNtLXkOXt8ch6SpgNHBTuh++kT7+UUn3KfmWPUvSlJJ+NpB0U3qc7pW0RUlcO0i6Ld03z0o6vGTZwZKeSZ83W9KpVe67yiKi4f4BLwJfAj4ErAY2KVl2NfAbYBgwAXgNuLtk+bEko+Bm4DRgHjAkXXYWcEV6e3OSbwTN6f07gJeADwBD0/vnpcvGAAuBg0g+DA9I72/Uw+27jWSU+jlgfIdl74srfezEdJ9sCbQA1wOXp8vGA8uAo4CB6bZPTJddCrwJTEr3x5XA1VXuq3uBKentFmCPCvGd0H4MSEamc9P2hqT3d+9kHwwD2oB9u9hXJ5CM4L+SxjkU2Do9BoOBjUg+bP4jXX8QMBM4Nd0fn0lfQ2eny/cFZqe3BwB/A76XPm9LktHigSWvl3fS494EnAvcVxLbK8D+FWJ/t68Oj38MWJPug2Ek397+Md2+XYAFwI7punOBvdPb6wO7pLcnAUvS/TCA5DW6Xbrst8BFadsbAw8AJ5Xsz9XAF9NtOhmYA6jkffCFCtt0IvD79PbRJO+Za0qW/a7K98AV6f69BPhi+tj1wBEkg7pj08e+ANyR3m4mee39DhhB8lp8s/0YlFn3tnS/bU7yHjohXf4BYL/0uG9M8n78UUl8syl5bQJbkLzPPpu2PZL33mdXpMdsV5LX3DW8l2eGk+So49LnfYgkd2ybLn8D+HB6e4P241tzDs2ikSz/AR9JX3gj0/vPAqemt5vSZduVrP8vlCT2TtpbBOxc8katlNj/ueR5XwL+lN7+NmkiLVl+M3B8D7dxfeA84CmS5PYosFtncaWP/Rn4Usn9bdP90Ax8F7ihTD+XAr8ouX8Q8GyV++ou4Aftx6Fknc7iO4H3EvtRwCNV7IOxaTulx/KHwGLgrfZjkbb9ahdtfaq9T+CjlCSq9LF76Dyx796x7XR//rLk9XJbybIdgLdL7r9CzxL7dum2jwGOBP7SYflFwPfT268CJwHrdrLOTzppexNgJTC05LGjgNtL9ueLJcvWSWMZVfI+qJTYt0qP0QDgZ2ls7fvzV8A3qnwPtCf2fYG/kCS1eSQf1l0l9j1K2rke+GaFdfcvWferwM1l4vkM8GDJ/Y6J/f8A/11hW35Wcv8Q4Mn09jHt+75k+cXAmentOWncw6vZb9X+a8SpmOOBWyJiQXr/17w3HbMRyQGbVbL+zNInSzot/WqzRNJikk/2kVRnXsntFSQjVUjmwo9QMtWxOG33I8CmHRtQUiHSfrKs07P5EbEoIr4TETuSvBEfBX5b4Svs6A7bOZNkP2wCjCMZNXV3m7raV58nGdU8K+lBSQdX6KNUV/G0W0Qyan13H0bE6ZHMs99Asn3tSo83kjaWdHU6zbCU5I3VHvdo4LVI3zWp971GSmwGjO5wXM8g2a/tOu6/Iap9nn8MSdJZnMawe4cYjgFGpeseTvKBPFPSnZL2TB8vt583Ixk1zi1p7yKSUela2xQRK9KbLVQhIl4ClgMTSaZtpgNzJG0L7APcWU07Je4k+ZA/g2S0v7KK55R9TXeiY64YDSBplJJptvbX0KVUzhM9fZ9tBuzV4fgeyXuv+8NIPgheVTIdvHuFPqrWUCeiJA0l+arTJKl9Rw0G1pO0M/AkydfycSQjeUimItqfvzfJ6Ho/4KmIWCNpEVDtnF85s0hG7F/sasWIuJJkyqMqEbFA0o9IPrw2IHnDdzSH5AXSbjzJfpifxjap2v7adbWvIuIF4Kh0TvLTwLWSNiwTX6lZJCPEiiLiLUn3p23f3tXqHe6fmz62U0QslPQpoH0+eC4wRpJKkvt4On9TzgJmRMQ2XcVbZVzVOgx4ON0Hs4A7I+KATjuIeBA4VMmJxVNIpiHHkcS+VSdPmUUyYh8ZEa09iK2abbqTZIQ7KCJek3QnyVTD+iSDlOo7iwhJV5Ik9k7Ph9VoHNBe3jqe5L0EyTmOlcDfRcSbkj4D/Kg0tA7tzAJ26kH/s4A/R5lqnYi4HzgkPb5fI5lq3qKzdbuj0UbsnyKZmtiBZEQwEdie5KvacZGUJl5PcgJtHUk78P6Tq8NJEt4bQLOk7wHrZhDXFSQnYQ6U1CRpiJKTcGN70pikf5U0IT0ZNJxknvPFiFiYxr6GZL633VXAqZK2kNRCMv10TfrGvRLYX9Jn0/Y2lDSxijAq7itJx0raKCLWkIwsITk2ncVXajowStLXJQ2WNLzCKOR04ERJ35G0cdrvWLp+YQ8nGTUuljQG+FbJsnvT7fpquj8+TfkPvgeApZK+LWloemwnqPpyxPmU3w/vo8QYSd8n+erdfkJ5OvABSVMkDUz/7SZpe0mD0m+AIyJiNbCU5BhA8nX+HyXtp+Tk+hhJ20XEXOAW4N8lrZsu20rSPhlu050kHzLtJ9LvIDkHcnf0rHz4J8ABEfHXHjy3K6dLWk/SeJKpmGvSx4eTTPktkTQO+GaH53XcD1cAkyUdnr6uRqaDza7cCOwo6eiS4ztJ0rbpa+5oSeumx3cZ7x3fmjRaYj+eZH7z1YiY1/6PZDR2TPoV+BSSrznzSL4+/bLk+TcDN5GUUc0kOfH1vq/xPRERs4BDSd6Mb6Rtfoue7791SKYbFpOcrNuMtKwz/Wp8DvDX9KvbHiQnmC4neSPNINmur6Trv0ryVf00khNJj1KmVLCDrvbVZOApScuB84HPRcQ7ZeJ7V0QsIzmh90mSY/QC8PedBRARd5OcSPwo8Hz6NfVPJIniggqx/4DkJOMS4A8kH/btba4i+RZwAsl0z5Glyzv035bGOZFkvy4gqVgZ0dn6nTgX+Od0P3RMDO1Gp/twOfAg8Hckc7e3pDEsAz5OciJ9Dsk++1eSb6oAU4BX0umCfyI54U1EPEBywvUn6X64k/e+1R1HclLw6XQfXEsn04ZlnA98Rh0qTjq4kyQxtif2u0le0+33kbSlkunI0V11GBELI+LPVcbXXb8neU88QvKeuzR9/Pu8dwL6RuC6Ds/7F+AH6bH9ekTMIHmtfJvkffYwybGsKCKWAAeSHLe5JMf3XN47vseTTLMtJZn+nNJZO93VfibczMwKotFG7GZmViMndjOzgnFiNzMrGCd2M7OCqU8d+/O3+IytmVl3rLMBjN21qr/JqU9iX+DLYZuZdcu6o2HsrlWt6qkYM7OCcWI3MysYJ3Yzs4JpqIuAmZn1pjWIt5o2oK15CLVfG7C3BE2t7zCs7U0G9PA6c07sZtZvvNW0AQNb1qNFbVT9O085i4CVMYS3lsPwtoU9asNTMWbWb7Q1D2FwAyd1AAkGqy39VtEzTuxm1o+ooZN6uyTGngfqxG5mVjBO7GZmOfvTX/7GtgedzNYHTuW8n1+beftO7GZmOWpra+PLZ1/ETRd9n6d//1Ou+uNdPP3iq5n24aoYM7NOTDr2TBYseXutx0eOGMoDV5zT43YfeOIFth6/KVuOS36v/HOf2Jvf/c/97LD1+C6eWT0ndjOzTixY8jY7nvSTtR5/6qJTa2r3tfkLGTdq5Lv3x44ayf2PZ3v9LE/FmJnlqLOfI1XGfyzlxG5mlqOxo0Yya96Cd+/PnreA0RtvkGkfTuxmZjnabcI2vDBzDjNmz2PVqtVcfdNfOOTvd8+0D8+xm5nlqLm5iQvPPIkDv3gWbWvWcOJh+7PjNtmdOAUndjOzTo0cMbTTE6UjRwytue2D9tmVg/ap7kczesKJ3cysE7WUNNab59jNzArGid3MrGCc2M3MCsaJ3cysYJzYzcwKxondzCxnJ555Pht/ZAoTDjmlV9p3Yjczy9kJh+3Hn6ad1WvtO7GbmVWwYNFSDj/l/7Jw8dLM2vzorhPYYERLZu115MRuZlbBZdffzKLXXuRX191c71Cq5sRuZlbGgkVLmX7r7fzXpzdh+q23Zzpq701O7GZmZVx2/c0cvJXYdpMhHLyV+syo3YndzKwT7aP14z60LgDHfWjdPjNqrzmxSxon6XZJz0h6StLXsgjMzKye2kfrI1uSayWObGnObNR+1Df/jT2POp3nXnmNsX//j1x83S01t1kqi6s7tgKnRcTDkoYDf5N0a0Q8nUHbZmZ1cccDjzFn7kp+/cTc9z0+esFjfOPzR9TU9lU/+lZNz+9KzYk9IuYCc9PbyyQ9A4wBnNjNrM+68aKz6x1Cj2U6xy5pc+CDwP2dLJsq6SFJD0373V+z7NbMzEpk9kMbklqA64CvR8RaZxciYhowDYB7Llj7Z7rNzHpdEAFSveOoLAKg52kykxG7pIEkSf3KiLg+izbNzLLW1PoOK6MpTZyNKQJWRhNNre/0uI2aR+ySBFwMPBMRP661PTOz3jKs7U3eWg7vNA8BGnXYHjS1LmNY25s9biGLqZi9gCnAE5IeTR87IyL+mEHbZmaZGUAwvG0htNU7kt6VRVXM3TTuR5+ZWb/jvzw160ULFi/n8O/8jIVL3qp3KNaPOLGb9aLL/nAPi+bN4lfTXeJr+XFiN+slCxYvZ/qdD/Jfnx7J9Dsf9KjdcuPEbtZLLvvDPRy89QC23XgwB289wKN2y40Tu1kvaB+tH7fLMACO22WYR+2WGyd2s17QPlp/35UBPWq3nGR2SQEze88dDz/PnNdX8usnXn/f46PnP883jvl4naKy/sKJ3awX3Pjvp9Q7hIa0YPFyTjrvCqZ9dwobjhhW73AKy1MxZpYbl3/mw4ndzHLh8s/8OLGbWS5c/pkfJ3Yz63Uu/8yXE7uZ9TqXf+bLVTFm1utc/pkvJ3Yz63Uu/8yXp2LM6uy5mfPZ4lNn8MKs17teuYd8+eD+xYndrM6+89Nr2aD5bU6/4L97rQ/Xj/cvTuxmdfTczPk88exL/PJTw3ji2Zd6ZdTu+vH+x4ndrI6+89NrOXpCMzuNGsjRE5p7ZdTu+vH+x4ndrE7aR+snTxoCwMmThmQ+anf9eP/kxG5WJ+2j9U1bmgDYtKUp81G768f7J5c7mtXJI8/N4oFVq7n4kcXve3zgoFmZ9eH68f5JEZF/r/dcUIdOzcz6sHVHw4TDVc2qnoqxPimLuuw8arur6SOPbal1eV5x5tVG0TmxW5+URV12HrXd1fSRx7bUujyvOPNqo+ic2K3PyaIuO4/a7mr6yGNbal2eV5x5tdEfOLFbn5NFXXYetd3V9JHHttS6PK8482qjP3Bitz4li7rsPGq7q+kjj22pdXlecebVRn/hxG59ShZ12XnUdlfTRx7bUuvyvOLMq43+wnXs1qdkUZedR213NX3ksS21Ls8rzrza6C9cx265W7B4OSeddwXTvjuFDUcMq3c4De+5mfOZ/LXzueWCr7PNuI3rHU5Z1RzXrtbJ4rVR2NeX69itkblcrXvyuKxvFrIomXQ5ZDac2C1XLlfrnjwu65uFLEomXQ6ZHSd2y5XL1bonj8v6ZiGLkkmXQ2Ynk8Qu6RJJr0t6Mov2rJhcrtY9eVzWNwtZlEy6HDJbWY3YLwUmZ9SWFZTL1bonj8v6ZiGLkkmXQ2Yrk3LHiLhL0uZZtGXF5XK17snjsr5ZyKJk0uWQ2cqs3DFN7NMjYkKZ5VOBqQAXnX7kh6Yeulcm/ZqZ9QuNWO4YEdMiYteI2NVJ3fLw3Mz5bPGpM8rOSd/3xAw2OOBUHnj6lbJtVLNOb8vikrpZXLY3izgtH66KscLqqv775B9ezubrBiede1nZNqpZp7flUR/u+vFicWK3Quqq/vu+J2awaOFCLjl0KIsWLux0RF7NOr0tj/pw148XT1bljlcB9wLbSpot6fNZtGvWU13Vf5/8w8s5ZqeBTBzVzDE7Dex0RF7NOr0tj/pw148XTyaJPSKOiohNI2JgRIyNiIuzaNesJ7qq/24fiX95t0EAfHm3QWuNyKtZp7flUR/u+vFi8lSMFU5X9d/tI/HRw5Plo4c3rTUir2ad3pZHfbjrx4vJl+21wumq/nvWvEVcPH8NFz+8+n3LW2PRu7erWae35VEf7vrxYvJle83M+oJGrGM3y1setdld1cpnwfXh1l1O7FZYedRm53GtdNeHW3c5sVsh5VGbnce10l0fbj3hxG6FlEdtdh7XSnd9uPWEE7sVTh612XlcK9314dZTTuxWOHnUZudxrXTXh1tPuY7dCieP2uw8rpXu+nDrKdexm5n1Ba5jt3pqlLrrPK5BbtaInNgtc41Sd53HNcjNGpETu2WqUequ87gGuVmjcmK3TDVK3XUe1yA3a1RO7JaZRqm7zuMa5GaNzIndMtModdd5XIPcrJG5jt0y0yh113lcg9yskbmO3d5nweLlnHTeFUz77hQ2HDGs03WemzmfyV87n1su+DrbjNs45wirV8221NpGHn2YAa5jt56rpgQwj0vVZiGPy/bm0YdZdzmx27uqKQHM41K1Wcjjsr159GHWE07s9q5qSgDzuFRtFvK4bG8efVj/8vqiZfzt2Vc7/ffcjNeqbscnTw14b+T4m88OB5ISwM/+5kGOP3ivd+d920fr///EFiC5VO3elySj9kaaa69mW2ptI48+rFiWLH+bi/70GG1rOl8ea4LH3whG7bxPp8s3a96Ebavsy4ndgMolgO2VIpUuVXvDD79ct9g7qmZbam0jjz6s7/jFzY/zxJwVFdd5c0Urf3fYV1lnnZay63xsWAtS5+dHR40YUnU8TuwGVFeqmMelarOQx2V78+jDeldE8OzM+bStKTOEBpatWMl/3voSQ0dsVLGtkdtNYsKU/bMOscdc7mhmhXTbIzN44pUFZZc/P2cRa8ZNYsiwdcuuEwOa2G73/Rg4aHBvhNgto0YM4ZM7j66q3NEjdjPrUxYueYsfXPMgAwaWT7ZtbWto22QHtvrI58qus8PgIQypMC3Slzmxm1lu5r+5lGUrVpZdfv19L/PMomaamsoX7LXSxC5Hfo+hw4b3RoiF4MRuZjVbvmIlv77jKcrPVsOipW/z5LIW1h+3Tdl1Ntju03x4u4nZB9jPOLGbWUVX3v40f3tlccV15i9dybb/8E8MGrpO2XWaBgxg341GZR2edcKJ3aygWlvbmPfm0rLLl771Dufd+DSD192gfCMB6245ke2nHFyxr+17GqT1Cid2sz7ovmdm89zshRXXuf2peQzZ4kMMUOfz1aF1mPT5cxg8ZGhvhGh15MRu1kAWL1vBOf/9IGvUVHad1a1tvL3e1my261EV2/rgbiNZp8UnGPsjJ3azjCxfsZJ3Vq0uuzwi+LffPsKSKF9i93brGnY54kyGDR/RGyFaP+HEbtaFt1euYvr9L1Dpb/kWLF3BnTPbGDFqfMW2xu/zRbYZv3XGEZq9XyaJXdJk4HygCfhFRJyXRbtmve239z7P3c/Or7jOnEVvs9l+JzCowlz0gDFN7Dd567LX+TDLU82JXVIT8FPgAGA28KCkGyPi6VrbNisnInjr7VUV13ly5hv87M8vMXRY51MfEcHgMTswYcqpFdup9op6Zo0iixH7JODFiHgZQNLVwKGAE7v1yJMvz2XGvMp109Mfnk3rBltBhRHy4Jb12HPqeQwY4J8dsP4li8Q+Bii9vN9sYPeOK0maCkwFuOj0I5l66F4ZdG19yfIVKznv2vtZ1VZ+ndWtbSxeZzyb7nRAxbY+cOQY1l1/ZMYRmhVDFom9syHTWqeZImIaMA3w1R37oLa2NVS6EmgAZ19zPwtWDSy7ztJ32pj4mW85IZv1siwS+2xgXMn9scCcDNq1HKxa3cpfH3+ZNRWS9vwl73DdY4sYvmHlPwffco+jmbjVjlmHaGbdlEVifxDYRtIWwGvA54CjM2jXahQR/PxPj/LSvPJ/Vv7qghVsstcRDBxcvuKjeeRADvzKRFd8mPURNSf2iGiVdApwM0m54yUR8VTNkVlFT86Yx3/c9CxDhpb/bcyVq1ax0a7/wNYHlD+f4Ypqs+LJpI49Iv4I/DGLtopuxpyFzFmwpOI6V98zgyUDN+787EVqUMv67Pb58xjQVP5Pz82sf/Jfnmbo1fmLuPjW8l9WVq5uZfaakWy8w54V2xl7yOHstPHorMMzs37Cib0KEcG5197P3OWV55iXtg5k1yPPpKm5fGXIVg3w24lmVmyFTuytrW089uJrFdeZu2gFv7p3DsPWq1CCF8H43Q9jp+12yThCM7Ps9enEfs2dT/Hy/GVllz89exHr7XwgTRV+9LapeRAHfOUbrvgws8JoyMT+wuw3+PH0pxk0uHxCfmflKobv+DHGffzDZdfZc9AgBnrqw8z6mbok9hMveazyCoNb2O2Ec2iuMFdt+Tv3lKNYvnztb0gtLcP57oVX1SEiM+tMXRL7nsedUY9urUbLly9jyy9csNbjL//iK3WIxszK8WXvzMwKxondzKxgnNjNzArGid3MrGAastzRGlNLy/BOT5S2tAyvQzRmVo4Tu1XNJY1mfYMTu2XGde5mjcGJ3TLjOnezxuCTp2ZmBePEbmZWME7sZmYF48RuZlYwPnlqmXGdu1ljcGK3qnVVzjj31ZdpXRNrLV/25huZxXDygROJTn7AW21t/NfNj2bWj0s3rS9zYreqdVXO2Lom2OyUy9ZaPvPC4zKLIZqaGHfKFWs9PuvCYzPrA1y6aX2b59jNzArGid3MrGCc2M3MCsaJ3cysYHzy1KrWVTlj8wB1eqK0eYAyi0FtbZ2eKFVbW2Z9gEs3rW9TxNrlab3t53e9nH+nZmZ92KgRQ/jkzqOrGiV5xN4AuqqZzqKmOo82vnrI7p3WsTcPEP954/1V9dEoXMdufZkTewPoqmY6i5rqPNrIo449L65jt77MJ0/NzArGid3MrGCc2M3MCsaJ3cysYGo6eSrpCOAsYHtgUkQ8lEVQ/U1XNdNZ1FTn0UYedex5cR279WU11bFL2h5YA1wEfLPaxO46djOz7smtjj0ingGQ+t6ILC9ZXD+8qzaqqR/vqo1q6rZP+vhOqGngWutE22ouuuXxTOLMolY+j3p617lbI8utjl3SVGAqwLGnnc1HDzkqr67rKovrh3fVRjX14121UU3dtpoGMu4rnbRxwbGZxZlFrXwe9fSuc7dG1mVil3QbMKqTRWdGxO+q7SgipgHTwFMxZma9qcvEHhH75xGImZllw+WOZmYFU1Nil3SYpNnAnsAfJN2cTVhmZtZTtVbF3ADckFEshZTF9cO7aqOa+vGu2qimbjvaVr97orRUtK3OLM4sauXzqKd3nbs1Ml+PvUZ9peytqzizKMusNQYzK8/XY89RXyl76yrOLMoya43BzLLhk6dmZgXjxG5mVjBO7GZmBePEbmZWMD55WqO+UvbWVZxZlGXWGoOZZcPljmZmfUB3yh09FWNmVjBO7GZmBePEbmZWME7sZmYF48RuZlYwTuxmZgXjxG5mVjBO7GZmBePEbmZWME7sZmYF48RuZlYwTuxmZgXjxG5mVjBO7GZmBePEbmZWME7sZmYF48RuZlYwTuxmZgXjxG5mVjBO7GZmBePEbmZWME7sZmYF48RuZlYwTuxmZgXjxG5mVjBO7GZmBVNTYpf0b5KelfS4pBskrZdVYGZm1jO1jthvBSZExE7A88B3aw/JzMxqUVNij4hbIqI1vXsfMLb2kMzMrBZZzrGfCNxUbqGkqZIekvTQXTdelWG3ZmZWqrmrFSTdBozqZNGZEfG7dJ0zgVbgynLtRMQ0YBrAz+96OXoUrZmZdanLxB4R+1daLul44GBgv4hwwjYzq7MuE3slkiYD3wb2iYgV2YRkZma1qHWO/UJgOHCrpEcl/SyDmMzMrAY1jdgjYuusAjEzs2z4L0/NzArGid3MrGCc2M3MCqamOfaeGjl8UD26NTPrs9aK1N5KAAAChUlEQVRbZ2DV68ql59WRNDX9I6tCKeJ2FXGbwNvV19RzuzwVU72p9Q6glxRxu4q4TeDt6mvqtl1O7GZmBePEbmZWME7s1SvcHGCqiNtVxG0Cb1dfU7ft8slTM7OC8YjdzKxgnNjNzArGib0bivjj3ZKOkPSUpDWSdq13PLWSNFnSc5JelPSdeseTBUmXSHpd0pP1jiUrksZJul3SM+nr72v1jikLkoZIekDSY+l2/aAecTixd08Rf7z7SeDTwF31DqRWkpqAnwKfAHYAjpK0Q32jysSlwOR6B5GxVuC0iNge2AP4ckGO1UrgYxGxMzARmCxpj7yDcGLvhiL+eHdEPBMRz9U7joxMAl6MiJcjYhVwNXBonWOqWUTcBbxZ7ziyFBFzI+Lh9PYy4BlgTH2jql0klqd3B6b/cq9QcWLvuYo/3m11MQaYVXJ/NgVIFkUnaXPgg8D99Y0kG5KaJD0KvA7cGhG5b1ddLgLWyLL68e5GUs02FYQ6ecz1vA1MUgtwHfD1iFha73iyEBFtwMT0HNwNkiZERK7nR5zYOyjij3d3tU0FMhsYV3J/LDCnTrFYFyQNJEnqV0bE9fWOJ2sRsVjSHSTnR3JN7J6K6YaSH+8+xD/e3ZAeBLaRtIWkQcDngBvrHJN1QpKAi4FnIuLH9Y4nK5I2aq+WkzQU2B94Nu84nNi7p3A/3i3pMEmzgT2BP0i6ud4x9VR6YvsU4GaSk3G/iYin6htV7SRdBdwLbCtptqTP1zumDOwFTAE+lr6XHpV0UL2DysCmwO2SHicZaNwaEdPzDsKXFDAzKxiP2M3MCsaJ3cysYJzYzcwKxondzKxgnNjNzArGid3MrGCc2M3MCuZ/AXRvB4wgYcACAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl0nPV97/H3d2a0r/Zo8SZbljCWa4JtEGCZ0BA77SUJF5o0bZNmb1KaNE3S9ObS0JyT3vb09qY3Pc3S3JNcSm6zQNOFJoXShCaBpIRiAzYIY7AxXrGxZS22LFmy9u/9Yx6NB1mbl9Ezo/m8zpkz2zMzH/tI+syz/H6PuTsiIiIAkbADiIhI5lApiIhIkkpBRESSVAoiIpKkUhARkSSVgoiIJKkUREQkSaUgIiJJKgUREUmKhR3gQlVVVXl9fX3YMUREssqOHTs63b16puWyrhTq6+vZvn172DFERLKKmR2ezXLafCQiIkkqBRERSVIpiIhIkkpBRESSVAoiIpKkUhARkSSVgoiIJOVMKew90cufPfQiA8OjYUcREclYOVMKR0/1c8/jB3nm8Kmwo4iIZKycKYXr6hcSjRhbD3SFHUVEJGPlTCmUFebxuqUVPLFfpSAiMpWcKQWAlsY4zx3ppm9wJOwoIiIZKbdKoSHOyJizXfsVREQmlVOl0Fy/gLyo8cT+zrCjiIhkpJwqheL8GOvrKtmm/QoiIpPKqVKAxCak5189Tc/AcNhRREQyTu6VQmMVYw5PHTgZdhQRkYyTc6WwYXkl+bGIxiuIiEwi50qhMC/KtcsXsFX7FUREzpNzpQCJ8Qq723o41TcUdhQRkYySk6WwqTGOOzx5UGsLIiKpcrIUrl5WSVFeVJuQREQmyMlSyI9FaK5foJ3NIiIT5GQpAGxqrGLviTN09A6GHUVEJGPkbCm0NMYB2Ka1BRGRpJwthauWlFNaENMmJBGRFDlbCrFohOtXLtTOZhGRFDlbCpA4NPVgZx9tpwfCjiIikhFyuhQ2NiT2K2w9oKm0RUQgzaVgZpVmdr+Z7TGz3WbWMuF5M7OvmNk+M9tpZtekM89Ev7C4nIqiPG1CEhEJxNL8/l8GHnb3d5hZPlA84fk3A6uCyw3A14LrORGJGBsbFuq8zSIigbStKZhZOfCLwDcA3H3I3bsnLHY78G1P2AZUmtnidGWaTEtDnKOnznLkZP9cfqyISEZK5+ajBqAD+Fsze9bM7jGzkgnLLAWOpNw/Gjw2Z1oaqwB0aKqICOkthRhwDfA1d98A9AGfmbCMTfI6n/iAmd1hZtvNbHtHR8dlDXllbSnxknztVxARIb2lcBQ46u5PBvfvJ1ESE5epS7m/DDg28Y3c/W53b3b35urq6ssa0szY2Bhn6/4u3M/rIxGRnJK2UnD3NuCIma0OHtoCvDhhsQeB9wVHIW0ETrv78XRlmkpLQ5y2ngEOdWm/gojktnQfffRx4L7gyKMDwAfN7CMA7v514AfAW4B9QD/wwTTnmdT4PEhP7O9kZdXE3R4iIrkjraXg7q1A84SHv57yvAMfS2eG2WioKqG2vICt+7t49w0rwo4jIhKanB7RPM7MaGmIs+3ASe1XEJGcplIItDTG6TwzyL72M2FHEREJjUoh0NKQGK+g0c0ikstUCoG6hUUsrSzSeAURyWkqhYCZ0dIYZ9vBLsbGtF9BRHKTSiFFS0Oc7v5hdrf1hB1FRCQUKoUU4+MVtAlJRHKVSiHFksoi6uPFbNPkeCKSo1QKE7Q0xnnywElGRsfCjiIiMudUChO0NFbROzjCC8e0X0FEco9KYYKNDQsBnV9BRHKTSmGCmrJCrqgp1c5mEclJKoVJtDTEefrQSYa1X0FEcoxKYRKbGuP0D42y8+jEU0qLiMxvKoVJ3NCg8QoikptUCpNYWJJP06IyTY4nIjlHpTCFTY1V7Dh8isGR0bCjiIjMGZXCFFoa4wyOjPHsK9qvICK5Q6UwhetXLiRiOr+CiOQWlcIUKoryWLukgm0qBRHJISqFaWxqjPPskVOcHdJ+BRHJDSqFaWxsjDM86uw4fCrsKCIic0KlMI3r6hcSjRhP7O8MO4qIyJxQKUyjtCDGumUVmhxPRHKGSmEGLY1xdh49zZnBkbCjiIiknUphBi0NVYyOOU8fPBl2FBGRtFMpzODaFQvIi5o2IYlITlApzKAoP8qG5Qs0OZ6I5ASVwiy0NMTZdew0p/uHw44iIpJWKoVZaGmM4w5PHtTagojMb2ktBTM7ZGbPm1mrmW2f5Pmbzex08HyrmX0unXku1obllRTEItqvICLzXmwOPuON7j7d6K+fu/utc5DjohXEojTXa7+CiMx/2nw0Sy0Ncfa09dJ1ZjDsKCIiaZPuUnDgR2a2w8zumGKZFjN7zsx+aGZr05znorU0VgHwpMYriMg8lu5SuNHdrwHeDHzMzH5xwvPPACvcfR3w18C/TPYmZnaHmW03s+0dHR3pTTyFq5dVUJwf1SYkEZnX0loK7n4suG4Hvg9cP+H5Hnc/E9z+AZBnZlWTvM/d7t7s7s3V1dXpjDylvGiE6+oXanI8EZnX0lYKZlZiZmXjt4FfBnZNWGaRmVlw+/ogT8Z+FW9pjLO/o4/2noGwo4iIpEU61xRqgcfN7DngKeDf3P1hM/uImX0kWOYdwK5gma8A73R3T2OmS7KpMQ6gQ1NFZN5K2yGp7n4AWDfJ419Puf1V4KvpynC5rV1SQVlhjK37u7h9/dKw44iIXHY6JPUCRCPGDSsXak1BROYtlcIFamms4nBXP8e6z4YdRUTkslMpXKCWhmC/gg5NFZF5SKVwgZoWlbGgOI8nVAoiMg+pFC5QJGLcsDLOtgNdZPCBUiIiF0WlcBE2XRHn1e6zHDmp/QoiMr+oFC7C+H4FjW4WkflGpXARrqgppaq0QIemisi8o1K4CGZGS2Ocrfu1X0FE5heVwkVqaYjT3jvI/o6+sKOIiFw2KoWL1KJ5kERkHlIpXKT6eDGLygvZpvEKIjKPqBQukpmxqTExXmFsTPsVRGR+UClcgo2Ncbr6htjb3ht2FBGRy0KlcAk0D5KIzDcqhUtQt7CYuoVFKgURmTdUCpdoU0MVWw90MTw6FnYUEZFLplK4RDevrqZ3YIQdh0+FHUVE5JKpFC7RTVdWkxc1Ht3THnYUEZFLplK4RKUFMTY2xHlk94mwo4iIXDKVwmWwuamG/R19HOrUlBcikt1mVQpm9p3ZPJarNjfVAGgTkohkvdmuKaxNvWNmUeDayx8nO62Il3BFTalKQUSy3rSlYGZ3mVkvcLWZ9QSXXqAdeGBOEmaJLU01PHmwi96B4bCjiIhctGlLwd3/l7uXAV9w9/LgUubucXe/a44yZoXNTTUMjzqPv6yzsYlI9prt5qOHzKwEwMzeY2Z/ZWYr0pgr61y7YgHlhTEe0SYkEclisy2FrwH9ZrYOuBM4DHw7bamyUCwa4ebVNfx0T7tmTRWRrDXbUhjxxHknbwe+7O5fBsrSFys7bVlTQ1ffEM8d7Q47iojIRZltKfSa2V3Ae4F/C44+yktfrOz0hiuriRg8slubkEQkO822FH4DGAR+y93bgKXAF9KWKktVFufTvGKh9iuISNaaVSkERXAfUGFmtwID7q59CpPYvKaG3cd7ONZ9NuwoIiIXbLYjmn8deAr4NeDXgSfN7B2zeN0hM3vezFrNbPskz5uZfcXM9pnZTjO75kL/AZnmTWs0ullEsldslst9FrjO3dsBzKwa+Alw/yxe+0Z3n+rg/TcDq4LLDSSOcrphlpkyUmN1KcsXFvPonnbes1FH7YpIdpntPoXIeCEEui7gtdO5Hfi2J2wDKs1s8WV439CYGZubavjPfZ2cHRoNO46IyAWZ7R/2h83s383sA2b2AeDfgB/M4nUO/MjMdpjZHZM8vxQ4knL/aPBYVtuypobBkTGe2K/RzSKSXabdfGRmVwC17v7fzeztwOsBA7aS2PE8kxvd/ZiZ1QA/NrM97v5Y6kdM8przRn4FhXIHwPLly2fxseG6fuVCSvKjPLKnnS1rasOOIyIyazOtKXwJ6AVw9++5+x+4+6dIrCV8aaY3d/djwXU78H3g+gmLHAXqUu4vA45N8j53u3uzuzdXV1fP9LGhK4hFuWlVNY/ubicx5k9EJDvMVAr17r5z4oPuvh2on+6FZlZiZmXjt4FfBnZNWOxB4H3BUUgbgdPufny24TPZ5jU1tPUM8OLxnrCjiIjM2kxHHxVO81zRDK+tBb5vZuOf83fu/rCZfQTA3b9OYo3jLcA+oB/44GxCZ4M3rg4OTd3dztolFSGnERGZnZlK4Wkz+213/5vUB83sQ8CO6V7o7geAdZM8/vWU2w58bPZxs0d1WQHr6ip5ZE87H9+yKuw4IiKzMlMp/D6Jb/vv5lwJNAP5wNvSGWw+2NJUwxd/speO3kGqywrCjiMiMqOZTrJzwt03AX8CHAouf+LuLcHUFzKNzU01uMPPXtLoZhHJDrMa0ezuPwV+muYs887aJeXUlhfw6J52fq25buYXiIiE7HKMSpYpJEY31/LY3g4GRzS6WUQyn0ohzd60poa+oVGeOngy7CgiIjNSKaTZpsYqCmIRnXhHRLKCSiHNivKj3HhFFY/sOaHRzSKS8VQKc2BzUw1HTp5lf8eZsKOIiExLpTAHNjclRjdrE5KIZDqVwhxYUlnEmsXlOneziGQ8lcIc2dJUw47Dp+juHwo7iojIlFQKc2TzmhpGx5z/2NsRdhQRkSmpFObIumWVxEvyeVSbkEQkg6kU5kg0Yty8uoafvdTByOhY2HFERCalUphDW9bUcPrsMM+80h12FBGRSakU5tBNq6qIRYxH9pwIO4qIyKRUCnOorDCPGxoW8qjGK4hIhlIpzLEtTbW83H6GV7r6w44iInIelcIc27ImOHezNiGJSAZSKcyxFfESGqtLNLpZRDKSSiEEW9bUsu1AF2cGR8KOIiLyGiqFEGxuqmF41Hn8ZY1uFpHMolIIwbUrFlBeGNOsqSKScVQKIciLRnjD6hp++lI7Y2M68Y6IZA6VQki2NNXQeWaIna+eDjuKiEiSSiEkb7iymojBo7t1aKqIZA6VQkgWlORz7YoFOjRVRDKKSiFEm5tqeeFYD22nB8KOIiICqBRCdW50s9YWRCQzqBRCtKqmlGULijTlhYhkjLSXgplFzexZM3tokuc+YGYdZtYaXD6c7jyZxMx405paHt/XycDwaNhxRETmZE3hk8DuaZ7/B3dfH1zumYM8GWVzUw0Dw2Ns3d8VdhQRkfSWgpktA94K5Nwf+9m6oWEhxflRnXhHRDJCutcUvgTcCUx3UuJfNbOdZna/mdWlOU/GKYhFuWlVFY/ubsddo5tFJFxpKwUzuxVod/cd0yz2r0C9u18N/AT41hTvdYeZbTez7R0d828SuS1NtRw7PcDu471hRxGRHJfONYUbgdvM7BDw98BmM7s3dQF373L3weDu3wDXTvZG7n63uze7e3N1dXUaI4fj5qbEv0lHIYlI2NJWCu5+l7svc/d64J3Ao+7+ntRlzGxxyt3bmH6H9LxVU1bIumUVGt0sIqGb83EKZvanZnZbcPcTZvaCmT0HfAL4wFznyRSbm2ppPdJN55nBmRcWEUmTOSkFd/+Zu98a3P6cuz8Y3L7L3de6+zp3f6O775mLPJloy5oa3OFnL82/fSYikj00ojlDrF1STm15gfYriEioVAoZwszY3FTDY3s7GRqZ7gheEZH0USlkkM1NtZwZHOHpQyfDjiIiOUqlkEFuvCJOfiyiczeLSGhUChmkOD/GjY1xHtlzQqObRSQUKoUMs3lNLYe7+jnQ2Rd2FBHJQSqFDLO5KTjxjjYhiUgIVAoZZmllEU2LyjRrqoiEQqWQgbasqeHpQ6c4fXY47CgikmNUChloc1Mto2POY3s1ullE5pZKIQOtr6skXpLPt7ceYnhUA9lEZO6oFDJQNGJ89q1rePrQKf7soRfDjiMiOSQWdgCZ3NuvWcaetl7ufuwATYvLedf1y8OOJCI5QGsKGewPb2niDVdW87kHdvHUQU19ISLpp1LIYNGI8ZV3baBuQTEfvXcHR0/1hx1JROY5lUKGqyjK42/e38zQ6Bh3fHsH/UMjYUcSkXlMpZAFGqtL+cq7NrCnrYdP/9NzmhdJRNJGpZAl3ri6hrvevIYfPN/GXz+6L+w4IjJP6eijLPLhm1ay+3gPf/XjvVxZW8YtVy0KO5KIzDNaU8giZsafv/11rKur5A/+sZU9bT1hRxKReUalkGUK86Lc/d5rKS2I8dvf3s7JvqGwI4nIPKJSyEK15YXc/b5mTvQM8rv37dBUGCJy2agUstT6ukr+4ldfx7YDJ/nTf9VUGCJyeWhHcxZ724Zl7Dney/997ABNi8t49w0rwo4kIllOawpZ7s5bmrh5dTV//MALPHmgK+w4IpLlVApZbnwqjOXxYj563zOaCkNELolKYR4oL8zjnvc1Mzw6xoe/tZ2+QU2FISIXR6UwTzRUl/LV37yGvSd6+fQ/PcfYmKbCEJELp1KYR95wZTV/9JY1/HCXpsIQkYujo4/mmQ+9fiUvHu/hiz/Zy+pFpdxy1eKwI4lIFtGawjxjZvz5217H+rpK/uAfn2P3cU2FISKzl/ZSMLOomT1rZg9N8lyBmf2Dme0zsyfNrD7deXLB+FQYZYUxPvyt7XSdGQw7kohkiblYU/gksHuK5z4EnHL3K4AvAn8xB3lyQk15IXe/t5mOM4P87n3PaCoMEZmVtJaCmS0D3grcM8UitwPfCm7fD2wxM0tnplyyrq6S//2rV/PkwZP8yb++EHYcEckC6V5T+BJwJzDV19SlwBEAdx8BTgPxiQuZ2R1mtt3Mtnd0dKQr67z0KxuW8jtvaODeba9w77bDYccRkQyXtlIws1uBdnffMd1ikzx23gH27n63uze7e3N1dfVly5gr7vwvTbxxdTX/48EX2KapMERkGulcU7gRuM3MDgF/D2w2s3snLHMUqAMwsxhQAZxMY6acFI0YX37XBlbEi/novTv40QttDI1oH4OInC9tpeDud7n7MnevB94JPOru75mw2IPA+4Pb7wiW0VDcNCgvzOOe919HfizCHd/ZwXX/8yfc9b2dbN3fpdHPIpI054PXzOxPge3u/iDwDeA7ZraPxBrCO+c6Ty5ZWVXC43+4mcdf7uSB1ld5oPUY333qCIvKC7lt/RJuW7eEtUvK0b5+kdxl2fbFvLm52bdv3x52jHmhf2iEn+xu58HWV/nZSx2MjDmN1SXcvn4pt61bQn1VSdgRReQyMbMd7t4843IqBQE41TfED3e18UDrqzx5MLFbZ11dJbevW8Kt6xZTU1YYckIRuRQqBblox7rP8tDOYzzQeowXjvUQMdjUWMVt65dwy1WLKC/MCzuiiFwglYJcFvvae3mgNVEQr5zsJz8WYUtTDbevX8LNq2sozIuGHVFEZkGlIJeVu9N6pJsHWo/x0M7jdJ4ZpKwgxi1XLeL29UtpaYwTjWgHdS5yd/Z3nOGxvZ08sb+LMXcWVRSyqDxxqa0oZHFFIbXlhZQXxnQgwwVwd7r7hznY1cehzj4aqktZX1d5Ue+lUpC0GRkdY+uBLh5oPcbDu9o4MzjCwpJ81i4pZ3VtGasXlbFmcTlX1JRm1JrE6JhzrPssBzr7OH12mMHhUQZHxoLLKIPD524PJR8fS1kuuB5OuZ3y/OiYc0VNKRuWV7K+rpL1dQu4oqZ0Xpblyb4hHt/Xyc/3dvD4vk6Onx4AoD5eTFF+jBM9A5zsGzrvdUV5URZVFFJbXsDiiiJqywtZVF6QKJGKIhaVF1JVmk8smlsTOHf3D3Gws4/DXf0c7OzjUFACBzv76Bk4dybFD95Yzx//17UX9RkqBZkTA8Oj/HRPO4/saeeltl72nuhlMBgYFzGoryqhaVEZq2vLWb2ojKZFZSxfWEwkTX8o3Z2uvsQv2MGOPg509nGw8wwHOhK/cEMzTAyYH4tQEI1QkBehIBalIBZJPJaXuJ24RIPnU+7HImCw53gvrUe6OX12GIDSghhXL6sISqKS9csrs3Kn/eDIKM8c7ubnL3fw85c72XXsNO5QXhjj9auquGlVNa+/ooq6hcXJ1wwMj9LeM0hbzwBtPQOcOJ24bku5bu8dYHj0tX+DIgbVZQWJNY1gjaMmWMsozo9RUhB97XV+jOKCKCX5MQrzIhm7JnL67DCHkn/w+znU1ZcsgO7+4eRyZrCkooj6qmLq4yWsrCqhPl5CfVUxdQuLKYhd3BctlYKEYnTMOdTVx0ttvexp6+Wlth5eauvl8Ml+xn/UivKiXLmojKZgraJpUeI6Xlow68/pHxpJ/OHv7ONAR3Dd2cfBjjOv+WaVFzVWxEtoqCphZXXiuj5eQry0IPEHPfWPfzRyWcrK3TnY2cezr3TTeiRx2X28h5FgkODSyiLWL69kQ10lG5ZXsnZJRUatUcFrNwn9/OUOth04ydnhUWIR45rlC7hpVRU3XVnN65ZWXNKa0NhYosRPTCiLtp6Bc4+dHqB3lucdNyNREvlRSgqC65TSeM3jBTEKYhGiESNiRsQS5yMZvx0xw4LrSGT8/rnnplu+++wwhzv7kpt9DnX1n7fmtKSikPqqksQlfq4A6hYWp+XnQaUgGaV/aIS9J87wUltPUBaJS1fKL0pVaUGyIJoWldG0qJySgiiHulL+8AfXbT0Dr3n/pZVFrKwqSV4aqktoqCplSWVhRmyKGBgeZderp2k90p0si1e7zwIQixhrFpezPiiJ9XWVrKwqmfNvvOObhB4P1gbGNwk1VJUk1wY2NiykLISjz84OjXJmcIT+oRH6BkcT10Oj9A8G16mPn/f8CP1Do4lLsHzf4EiypNNpcUUhK+LFKd/2Ez+fy9P0h386KgXJCh29g8FaRU9y7SJ1E1SqyuK8xDf+qlIaqs8VQH28hKL8zPqmPRvtPQM8G6xJtL7Szc6j3fQNjQJQUZSX3OS0ZnE5RflR8qJGXjRCXjRCLDJ+O+Wx6Gsfi0Vs2mK5mE1C88nQyBhnh0dxd8YcxtwZc8eTtxNrMp7y3Jgz/fLuuDslBTFWLMysn0uVgmSt0THncLAJqm9oNPHNv6qEBSX5YUdLq9Ex5+X2XlpfObc2sbe9l0v5Fc2LGrHI+eWRH43Q1jNA/9C5TUKJIqji6mWV83LneK5TKYjMA2cGRzjY0cfQ6ChDI87I2BjDo2MMjzrDo2OMjDpDwfXI2BhDI2OMjDnDI2MMj40vc2751NfES/J5fYibhGRuzbYU5nxCPBGZvdKCGK9bVhF2DMkh4e+BExGRjKFSEBGRJJWCiIgkqRRERCRJpSAiIkkqBRERSVIpiIhIkkpBRESSsm5Es5l1AIcv8uVVQOdljJNu2ZQ3m7JCduXNpqyQXXmzKStcWt4V7l4900JZVwqXwsy2z2aYd6bIprzZlBWyK282ZYXsyptNWWFu8mrzkYiIJKkUREQkKddK4e6wA1ygbMqbTVkhu/JmU1bIrrzZlBXmIG9O7VMQEZHp5dqagoiITCNnSsHMbjGzl8xsn5l9Juw8UzGzOjP7qZntNrMXzOyTYWeaDTOLmtmzZvZQ2FmmY2aVZna/me0J/o9bws40HTP7VPBzsMvMvmtmhWFnSmVm/8/M2s1sV8pjC83sx2b2cnC9IMyM46bI+oXgZ2GnmX3fzCrDzJhqsrwpz33azNzMqi735+ZEKZhZFPg/wJuBXwDeZWa/EG6qKY0A/83d1wAbgY9lcNZUnwR2hx1iFr4MPOzuTcA6MjizmS0FPgE0u/tVQBR4Z7ipzvNN4JYJj30GeMTdVwGPBPczwTc5P+uPgavc/WpgL3DXXIeaxjc5Py9mVgf8EvBKOj40J0oBuB7Y5+4H3H0I+Hvg9pAzTcrdj7v7M8HtXhJ/tJaGm2p6ZrYMeCtwT9hZpmNm5cAvAt8AcPchd+8ON9WMYkCRmcWAYuBYyHlew90fA05OePh24FvB7W8BvzKnoaYwWVZ3/5G7jwR3twHL5jzYFKb4vwX4InAnkJYdwrlSCkuBIyn3j5Lhf2gBzKwe2AA8GW6SGX2JxA/pWNhBZtAAdAB/G2zqusfMSsIONRV3fxX4SxLfCI8Dp939R+GmmpVadz8OiS85QE3IeWbrt4Afhh1iOmZ2G/Cquz+Xrs/IlVKwSR7L6MOuzKwU+Gfg9929J+w8UzGzW4F2d98RdpZZiAHXAF9z9w1AH5mzaeM8wbb424GVwBKgxMzeE26q+cnMPkti0+19YWeZipkVA58FPpfOz8mVUjgK1KXcX0aGrYanMrM8EoVwn7t/L+w8M7gRuM3MDpHYLLfZzO4NN9KUjgJH3X18zet+EiWRqd4EHHT3DncfBr4HbAo502ycMLPFAMF1e8h5pmVm7wduBd7tmX2MfiOJLwjPBb9vy4BnzGzR5fyQXCmFp4FVZrbSzPJJ7Kx7MORMkzIzI7HNe7e7/1XYeWbi7ne5+zJ3ryfx//qou2fkt1l3bwOOmNnq4KEtwIshRprJK8BGMysOfi62kME7xlM8CLw/uP1+4IEQs0zLzG4B/hC4zd37w84zHXd/3t1r3L0++H07ClwT/FxfNjlRCsGOpN8D/p3EL9U/uvsL4aaa0o3Ae0l8424NLm8JO9Q88nHgPjPbCawH/jzkPFMK1mjuB54Bnifx+5pRI3DN7LvAVmC1mR01sw8Bnwd+ycxeJnGUzOfDzDhuiqxfBcqAHwe/a18PNWSKKfKm/3Mze21JRETmUk6sKYiIyOyoFEREJEmlICIiSSoFERFJUimIiEiSSkFyjpmdCa7rzew3L/N7/9GE+09czvcXSTeVguSyeuCCSiGYcXc6rykFd8+GEcgiSSoFyWWfB24KBi19KjgnxBfM7Olgfv3fATCzm4NzXPwdiUFkmNm/mNmO4FwHdwSPfZ7EjKatZnZf8Nj4WokF773LzJ43s99Iee+fpZzj4b5g9DJm9nkzezHI8pdz/r8jOSkWdgCREH0G+LS73woQ/HE/7e7XmVkB8J9mNj4r6fUk5t0/GNz/LXc/aWZFwNNm9s/u/hkz+z13Xz/JZ72dxAjqdUBV8JrHguc2AGtJzMf1n8CNZvYi8DZC+TJJAAABYUlEQVSgyd09k07+IvOb1hREzvll4H1m1kpiuvI4sCp47qmUQgD4hJk9R2IO/rqU5abyeuC77j7q7ieA/wCuS3nvo+4+BrSS2KzVAwwA95jZ24GMnpdH5g+Vgsg5Bnzc3dcHl5Up5y/oSy5kdjOJGUxb3H0d8Cww02kyJ5u+fdxgyu1RIBbM13U9idlyfwV4+IL+JSIXSaUguayXxGRo4/4d+GgwdTlmduUUJ+GpAE65e7+ZNZE4beq44fHXT/AY8BvBfotqEmeAe2qqYMH5NCrc/QfA75PY9CSSdtqnILlsJzASbAb6JonzN9eTmKPeSJylbbJTST4MfCSYafUlEpuQxt0N7DSzZ9z93SmPfx9oAZ4jcYKnO929LSiVyZQBD5hZIYm1jE9d3D9R5MJollQREUnS5iMREUlSKYiISJJKQUREklQKIiKSpFIQEZEklYKIiCSpFEREJEmlICIiSf8fFJT0ts86UgQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mlxtend.data import iris_data\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "from mlxtend.classifier import Adaline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Loading Data\n",
    "\n",
    "X, y = iris_data()\n",
    "X = X[:, [0, 3]] # sepal length and petal width\n",
    "X = X[0:100] # class 0 and class 1\n",
    "y = y[0:100] # class 0 and class 1\n",
    "\n",
    "# standardize\n",
    "X[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()\n",
    "X[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()\n",
    "\n",
    "\n",
    "ada = Adaline(epochs=15, \n",
    "              eta=0.02, \n",
    "              minibatches=5, # for SGD learning w. minibatch size 20\n",
    "              random_seed=1,\n",
    "              print_progress=3)\n",
    "\n",
    "ada.fit(X, y)\n",
    "plot_decision_regions(X, y, clf=ada)\n",
    "plt.title('Adaline - Stochastic Gradient Descent w. Minibatches')\n",
    "plt.show()\n",
    "\n",
    "plt.plot(range(len(ada.cost_)), ada.cost_)\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Cost')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "## Adaline\n",
      "\n",
      "*Adaline(eta=0.01, epochs=50, minibatches=None, random_seed=None, print_progress=0)*\n",
      "\n",
      "ADAptive LInear NEuron classifier.\n",
      "\n",
      "Note that this implementation of Adaline expects binary class labels\n",
      "in {0, 1}.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `eta` : float (default: 0.01)\n",
      "\n",
      "    solver rate (between 0.0 and 1.0)\n",
      "\n",
      "- `epochs` : int (default: 50)\n",
      "\n",
      "    Passes over the training dataset.\n",
      "    Prior to each epoch, the dataset is shuffled\n",
      "    if `minibatches > 1` to prevent cycles in stochastic gradient descent.\n",
      "\n",
      "- `minibatches` : int (default: None)\n",
      "\n",
      "    The number of minibatches for gradient-based optimization.\n",
      "    If None: Normal Equations (closed-form solution)\n",
      "    If 1: Gradient Descent learning\n",
      "    If len(y): Stochastic Gradient Descent (SGD) online learning\n",
      "    If 1 < minibatches < len(y): SGD Minibatch learning\n",
      "\n",
      "- `random_seed` : int (default: None)\n",
      "\n",
      "    Set random state for shuffling and initializing the weights.\n",
      "\n",
      "- `print_progress` : int (default: 0)\n",
      "\n",
      "    Prints progress in fitting to stderr if not solver='normal equation'\n",
      "    0: No output\n",
      "    1: Epochs elapsed and cost\n",
      "    2: 1 plus time elapsed\n",
      "    3: 2 plus estimated time until completion\n",
      "\n",
      "**Attributes**\n",
      "\n",
      "- `w_` : 2d-array, shape={n_features, 1}\n",
      "\n",
      "    Model weights after fitting.\n",
      "\n",
      "- `b_` : 1d-array, shape={1,}\n",
      "\n",
      "    Bias unit after fitting.\n",
      "\n",
      "- `cost_` : list\n",
      "\n",
      "    Sum of squared errors after each epoch.\n",
      "\n",
      "**Examples**\n",
      "\n",
      "For usage examples, please see\n",
      "    [https://rasbt.github.io/mlxtend/user_guide/classifier/Adaline/](https://rasbt.github.io/mlxtend/user_guide/classifier/Adaline/)\n",
      "\n",
      "### Methods\n",
      "\n",
      "<hr>\n",
      "\n",
      "*fit(X, y, init_params=True)*\n",
      "\n",
      "Learn model from training data.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "- `y` : array-like, shape = [n_samples]\n",
      "\n",
      "    Target values.\n",
      "\n",
      "- `init_params` : bool (default: True)\n",
      "\n",
      "    Re-initializes model parameters prior to fitting.\n",
      "    Set False to continue training with weights from\n",
      "    a previous model fitting.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `self` : object\n",
      "\n",
      "\n",
      "<hr>\n",
      "\n",
      "*get_params(deep=True)*\n",
      "\n",
      "Get parameters for this estimator.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `deep` : boolean, optional\n",
      "\n",
      "    If True, will return the parameters for this estimator and\n",
      "    contained subobjects that are estimators.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `params` : mapping of string to any\n",
      "\n",
      "    Parameter names mapped to their values.'\n",
      "\n",
      "    adapted from\n",
      "    https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py\n",
      "    Author: Gael Varoquaux <gael.varoquaux@normalesup.org>\n",
      "    License: BSD 3 clause\n",
      "\n",
      "<hr>\n",
      "\n",
      "*predict(X)*\n",
      "\n",
      "Predict targets from X.\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `target_values` : array-like, shape = [n_samples]\n",
      "\n",
      "    Predicted target values.\n",
      "\n",
      "<hr>\n",
      "\n",
      "*score(X, y)*\n",
      "\n",
      "Compute the prediction accuracy\n",
      "\n",
      "**Parameters**\n",
      "\n",
      "- `X` : {array-like, sparse matrix}, shape = [n_samples, n_features]\n",
      "\n",
      "    Training vectors, where n_samples is the number of samples and\n",
      "    n_features is the number of features.\n",
      "\n",
      "- `y` : array-like, shape = [n_samples]\n",
      "\n",
      "    Target values (true class labels).\n",
      "\n",
      "**Returns**\n",
      "\n",
      "- `acc` : float\n",
      "\n",
      "    The prediction accuracy as a float\n",
      "    between 0.0 and 1.0 (perfect score).\n",
      "\n",
      "<hr>\n",
      "\n",
      "*set_params(**params)*\n",
      "\n",
      "Set the parameters of this estimator.\n",
      "The method works on simple estimators as well as on nested objects\n",
      "(such as pipelines). The latter have parameters of the form\n",
      "``<component>__<parameter>`` so that it's possible to update each\n",
      "component of a nested object.\n",
      "\n",
      "**Returns**\n",
      "\n",
      "self\n",
      "\n",
      "adapted from\n",
      "https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py\n",
      "Author: Gael Varoquaux <gael.varoquaux@normalesup.org>\n",
      "License: BSD 3 clause\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.classifier/Adaline.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
